Participating in our project means making judgements about a published research claim.
What does that mean exactly?
Our project aim is to assess 3000 published research articles and 100 COVID-19 pre-prints in the following disciplines: criminology, economics, education, marketing, management, psychology, public administration, and sociology. The full list of journals the claims are drawn from is listed here.
What we ask you to do is to evaluate the replicability of a claim, that is if you were to follow the methods of the original study with a high degree of similarity do you think we’d find similar results to the original study?
Don’t worry, we don’t ask you to do this alone. Our method involves structured group discussion – each claim is assessed by 3-5 other people, and you get to see what others in your group say before submitting your final judgement.
Why get involved?
Be a part the largest effort to evaluate reliability in the social & behavioural sciences! You’ll also get to:
- improve your peer-review & error detection skills
- calibrate your judgements & reasoning against your peers
Who can assess claims? Every participant counts – don’t worry about being an expert, we need diverse views
Our method – the IDEA protocol – harnesses the power of structured group discussion in predicting likely replicability. We have built a custom web-based platform to do this. What we ask you to do is to evaluate the replicability of a claim, that is if you were to follow the methods of the original study with a high degree of similarity do you think we’d find similar results to the original study?
Part of the scope of the repliCATS project, and indeed the wider SCORE program is to examine the markers of expertise (e.g. education, experience, domain knowledge), and the role they may play in making good judgements about the likelihood a research claim will replicate.
This means an eligible research participant for our project is someone who has completed or is completing a relevant undergraduate degree, and is over 18 years of age. And, importantly, is interested in making judgements about replicability.
If you would like more information, you can:
- Watch this short video demo of the platform on our resources page.
- Check out what other participants have said about getting involved.
How to get started on our remote platform
You can assess claims anywhere, anytime!
To create an account, or to login visit: https://score.eresearch.unimelb.edu.au
We have a custom-built platform participants use to assess claims. Every month, we release a mix of new claims across the disciplines that are in scope for the SCORE program. (The first time you create an account, you will be asked to complete a consent form, as well as a short demographics survey and a quiz. All up this takes about 30 minutes.)
As of August 2020, we have completed assessment of the first 3000 claims.
- Read more about: repliCATS monthly prizes and rules
- View: Monthly winners! (Of repliCATS remote rounds 1-7 which ran from Jan – Jul 2020)
Express interest in assessing COVID-19 claims
The repliCATS project — and the wider SCORE program — has been expanded by DARPA to include 100 COVID-19 research claims from the social and behavioural sciences. Click here to read more and express interest.
Just want to stay up-to-date on the project? Subscribe to our newsletter
We have a quarterly newsletter we send out about our project. By subscribing you’ll get a short, snappy newsletter letting you know what we’ve been up to, and what’s happening with the repliCATS project.
Privacy Collection Notice – the repliCATS project.
Human ethics application ID: 1853445.1
The information on this form is being collected by the repliCATS project, a research group at the University of Melbourne. You can contact us at repliCATSfirstname.lastname@example.org.
The information you provide will be used to communicate with you about the repliCATS project. The information will be used by authorised staff for the purpose for which it was collected, and will be protected against unauthorised access and use.
We’ve assessed the first 3000 claims!
Thank you to all our fantastic participants. The remote platform is now closed.
Express interest in COVID-19 claims workshop.
The repliCATS project will run a workshop from 2-8 September to assess 100 COVID-19 research claims from the social and behavioural sciences. There are 100 x USD200 (AUD 280) assessment grants available.