November workshop – 18-25 Nov 2021

Hi folks, this post has all the info you need if you are participating in the repliCATS pre-AIMOS workshop running from 18-25 November. A special acknowledgement to AIMOS, this is the second workshop we’ve run before their annual conference.

Here are links to all the information you’ll lead leading up to and during the week:

—

About the workshop

The workshop will run from 18-25 November (for your timezone), and during that time you’ll be asked to assess 2-3 bushel papers from economics, marketing, maanagement and/or adjacent disciplines. You’ll work in small groups of 4-6 people, each with a dedicated facilitator.

Here’s how it will work:

  • Before kick-off – you’ll get invited to the workshop slack space, receive your repliCATS platform sign-up details, be introduced to your group members & facilitator, and confirm your banking details so we can pay you the US$200 grants! (30-40 mins)
  • 18 October – workshop kick-off & introduction session* (45 mins). You can view the intro on YouTube here: https://youtu.be/H1XraKzBGDY
  • 18 October to your scheduled discussion – complete your round 1 judgements (~2-2.5 hours, we don’t expect you to work on the weekends, but you have that extra time in case…)
  • 24 or 25 November (or maybe earlier to later for those who have a public holiday) – group discussion and submit your round 2 judgements (~1-1.5 hours). Group facilitators will contact you to schedule a synchronous virtual discussion at a time that suits your group.
  • We’ll have a dedicated Slack space for everyone — there will be some friendly comps, some #random replipets fun.
  • Wrap-up & your feedback.

*There will be two live sessions accommodating different time zones – you only need to turn up to one 🙂

—

Things to do before the workshop

  • Let us know if your availability changes at any time.
  • Your banking details so we can pay you.
    • New participants: send us your first and last name as they appear on your bank account, and look for an e-mail from the University to set you up as a supplier.
    • Returning participants: you don’t need to do anything unless your bank details have changed.
  • Save the dates for the workshop introductions. These introductions will take you through everything you need to know! If you are a return participant, you don’t have to attend the workshop intro … unless you want a refresher! We will also record the intro and post it in the Slack channel for you to watch later.
  • Accept invitation into the slack workspace. For the week, we will have a dedicated slack workspace. Please try to use the same email address as the one you used for your consent & demographics so we know it’s you! Don’t want to use slack? See our faqs below.
    —

Communication

👩🏻‍💻 repliCATS on call: Live Q&A troubleshooting assistance (AEST) on slack

We will have repliCATS staff members available during working hours throughout the week.

The slack channel is for communicating and troubleshooting (and fun) during the week, and is separate to the reasoning, discussion, and comments we ask you to provide while assessing claims on our repliCATS platform.

When you evaluate claims within the online platform you will have an avatar that protects your anonymity and the data are recorded for our project. On Slack, you are identifiable so you can get to know each other. We will not be recording your Slack conversations. We want to understand the way you think and reason about claims so we would really appreciate it if you keep any information about this on the repliCATS platform and for your discussion sessions.

If you have trouble with your Slack channel please contact us, repliCATS-project@unimelb.edu.au

#️⃣ twitter

You can tweet using #repliCATS2021 @repliCATS

Please don’t share estimates or claim information publicly, this helps us protect the integrity of the SCORE program.

😺 the repliCATS organising team

You can contact Ans, Rania and Mel via: repliCATS-project@unimelb.edu.au

—

Preparation & reference material

For workshop week, you will assess claims and follow an elicitation protocol termed the ‘IDEA protocol’. The following materials will help you to understand the protocol, why we are using it, and what we will ask of you during the workshop. Everything you need can be found via the Resources for participants page.

The link to the platform is: https://replicats-bushel.eresearch.unimelb.edu.au/

The following documents are available via the Resources for participants page and may be useful if read before the introduction to the workshop. They can be easily accessed via the platform by clicking the resources button 🎓 :

  • repliCATS_BushelGuide_2021, plus a guide to round 1 and guide to round 2
  • Participant cheat sheet
  • Code of conduct
    —

FAQs about the workshop

The following FAQs are specific to this virtual workshop. For general FAQs about answering claims, check the participant resources and the platform.

  • What if I don’t want to use Slack?
  • What can I discuss on Slack?
  • Who is in my group? And who is my facilitator?
  • How do I know when I have assessed all papers?
  • What if I become ill or cannot finish the workshop?
  • What if I can’t finish all papers on time?
  • What if I can’t attend my group’s discussion meeting?
  • I don’t like the claims assigned to my group.
  • I missed the introduction. Can I catch up?
  • Where or how can I give feedback?
  • What if I no longer want to take part?

What if I don’t want to use Slack?

We will be using slack to create a sense of fun and community as well as answering any questions you might have. However, if you do not want to use Slack you can send questions via email to repliCATS-project@unimelb.edu.au

You can also access the most important information you need to participate via the platform’s community page – there is a link to this web post. The community button (👥) is at the top right of your page once you have logged into the online platform. It is the 4th button along.

What can I discuss on Slack?

Discuss on Slack:

  • Questions about how to use the platform
  • Questions about the workshop generally
  • Clarifying the meaning of the questions

Save for the repliCATS platform & group discussion:

  • Descriptions of your reasons when evaluating a paper
  • Responses to other people’s justifications
  • Any information or discussion you might to have about a specific paper

How do I know when I have evaluated all the papers?

This should be really easy to track because each group of 4-6 people will be assigned 2-3 papers in total. We will ask you to do the following during the workshop:

  • Complete round 1 within the first six days
  • Complete discussion and round 2 in the last two days of the assessment period
  • Your facilitator will contact you with an update on your progress. We will contact you again after the assessment period has ended confirming that you have reached your goal.

You can also check the status of a paper at any time in the platform by hovering over the little harvey balls on the top right of each paper. It will tell you how many people have completed round 1. See question below.

How do I know how many people are already in the paper?

The easiest way to tell is to hover over the little pie chart  (or harvey ball) on the homepage of the platform. It looks like this.

It will tell you how many people are assessing the paper, and which round they are in. The first number will always be the number of people in your group, the second number is how many people have saved a round 1 response, the third number how many have submitted a round 1 response, the fourth number is how many people have submitted a round 2 response.

What if I become ill?

We’re sorry to hear that, and hope you have mild symptoms. Please let us know, and we’ll work out a way for you to keep your grant that doesn’t involve you assessing papers while you are unwell!

See: answer below.

What if I can’t finish the papers on time?

Don’t worry – we know that things don’t always go according to plan.

As much as possible, we’d like for everyone to assess the papers assigned to you in the week. If you think you’ll struggle to make it, please reach out to us on repliCATS-contact@unimelb.edu.au and we’ll work something out with you.

What if I can’t attend my group’s discussion meeting?

We’d really like you to try and make your group’s discussion. If you can’t attend ––that is a shame because participants have consistently told us that discussing papers is one of the most fun and enriching parts of our workshops. However, we understand that when you are all working virtually/from home lots of unanticipated things might come up.

If you can’t make the scheduled discussion time, that’s OK. Let your facilitator know as soon as possible. You can still update your round 2 after the end of the workshop — you will still have the benefit of reviewing your group’s responses and reasoning from round 1 before doing so.

I don’t like the papers assigned to my group.

We’ve tried our best to allocate papers that match the group’s expertise. We will not change the papers that have been assigned to your group except if you have been assigned a paper that you are working on a replication study for with TA1 or the Center for Open Science.

What if I no longer want to take part?

We’d be really sad to see you leave during assessment week, but we understand that things come up. Please let us know as soon as possible if you no longer want to take part, so we can offer the assessment grant to someone else on the waitlist.

I missed the introduction. Can I catch up?

Yes we will post the link to the recorded version in Slack and in the web post.

Where or how can I give feedback?

We’ll send you a link to an evaluation once the week is over. We’d love to know your thoughts and comments on the whole process.

Facilitator FAQs

Who’s in my group?

We have sent you an email with your group name and the link to the Facilitator Spreadsheet (also pinned to this post!). This contains all the information about your group and its members. If you can’t find the email, let us know via repliCATS-contact@unimelb.edu.au and we will resend you those details.

How do I contact my group members?

You will have a private group channel on Slack. If one or more of your group members does not want to use Slack, make sure you reach out to them via email to introduce yourself. They might also contact you via email.

I don’t know the answer to a question from one of my group members!

The #facilitator channel on Slack will be the hive mind of facilitators. Before you contact the repliCATS team, check the channel for info. It is likely that your question will have been posted before and answered. If not, ask us at repliCATS-contact@unimelb.edu.au.

When should I schedule my group’s discussion session?
People are busy, so try to settle on a date and time as soon as you can. We recommend scheduling the discussion for Monday 26 July or Tuesday 27 July to give people ample time to complete their Round 1 judgements.

What if someone can’t attend the discussion?

We encourage all group members to attend the discussion, but it might not be possible to find a date and time that suits everyone, and sometimes people’s plans change at the last minute (e.g. they might have responsibilities as a carer). It is helpful for us to know how many people attended the discussion, so please log attendance on the Facilitator Spreadsheet (pinned to the post in the slack channel).

I have never facilitated a discussion before – what do I do?

Your main role is to ensure that everyone gets a chance to talk and to keep the discussion focused on exploring the reasoning that underpins Round 1 judgements. For more detailed guidance, review the facilitator training (a recording can be found here) and the Guidelines for Facilitators (pinned to the post in the slack channel)

When does Round 2 start?

Whenever your group has discussed a paper, participants can start updating their estimates. Whether you want to do this in real-time, i.e. as you are discussing the questions, or after the discussion is finished, is up to you.

What happens at the end of the workshop?

Please be sure to thank your group on behalf of the repliCATS team. Stay tuned for a wrap-up event. It will be different and fun, but as usual, there will be awards! Remind your participants that it might be up to 6 weeks for their assessment grants to be processed.