Statistical Forensics

Preregistration and Open Data

General Information

Preregistration and open data allows researchers to publicly announce what experiments they will conduct ahead of time, to defeat publication bias, “journals’ favoring of certain types of results—and, consequently, the questionable research practices that some scientists adopt to make their results more likely to be published” (Pain, 2015). Preregistration and open data allows for increased transparency and reliability in a study. This also allows for a study to be more easily reproduced. This process includes submitting manuscripts on the hypotheses, proposed experimental methods, and methods of analysis. 

 

Preregistration and open data, although may slightly decrease the chances for publication, there are many benefits for a researcher to preregister and allow open data. Starting off, it alleviates the pressure for  new researchers or ‘junior researchers’ to get ‘good results.’ Preregistering sets new standards in terms of statistical power and methodological rigor and sends the signal that your work reaches the highest methodological standards, with potential benefits in terms of professional credibility and citations. There are many sites that allow you to preregister your work such as Open Science, IRB Preregistration, Center for Open Science, Registered Report, etc. With the ease and benefits of preregistering, those who don’t use preregistration and open data may have increased suspicion that their work contains fraudulent data, fraudulent responses, etc if their work is ever questioned. To learn more about preregistration and open data, please visit the sources below.


Here are some resources:

AsPredicted: Preregistration.

AEA RCT Registry: The American Economic Association’s registry for randomized controlled trials.

Alvarez, R.M. (2014). The pros and cons of research preregistration. OUPblog.

Bergkvist, L. (2020). Preregistration as a way to limit questionable research practice in advertising research. International Journal of Advertising, 1-9.

Bert, B., Heinl, C., Chmielewska, J., Schwarz, F., Grune, B., Hensel, A., … & Schönfelder, G. (2019). Refining animal research: The Animal Study Registry. PLoS biology, 17(10).

Bracken, M. B. (2011). Preregistration of epidemiology protocols: a commentary in support. Epidemiology, 22(2), 135-137.

Burlig, F. (2018). Improving transparency in observational social science research: A pre-analysis plan approach. Economics Letters, 168, 56-60.

Chambers, C. & Munafo, M. (2013). Trust in science would be improved by study pre-registration. The Guardian.

Chambers, C. (2017). How Registered Reports and Preregistration Benefit Early Career Researchers [Webinar]. Center for Open Science.

Claesen, A., Gomes, S. L. B. T., & Tuerlinckx, F. (2019). Preregistration: Comparing dream to reality. PsyArXiv.

COMET Initiative: Core Outcome Measures in Effectiveness Trials.

COMPARE: tracking switched outcomes in clinical trial

CONSORT: transparent reporting of trials.

Crystal, M. (2019). Preregistration can bring science back to its roots. PLoS Blogs.

DeHaven, A. (2017). 10 Tips for Making a Great Preregistration. Center for Open Science.

DeHaven, A. (2017). Preregistration: A Plan, Not a Prison. Center for Open Science.

DeHaven, A. (2018). One Preregistration to Rule Them All? Center for Open Science.

DeHaven, A. (2020). Answering Your Preregistration Questions. Center for Open Science.

Dirnagl, U. (2020). Preregistration of exploratory research: Learning from the golden age of discovery. PLoS Biology, 18(3), e3000690.

EGAP: evidence in governance and politics. Preregistration.

Einarsdóttir, G. (2018). Why pre-registration might be better for your career and well-being. Center for Open Science.

Fraser, H., Parker, T., Nakagawa, S., Barnett, A., & Fidler, F. (2018). Questionable research practices in ecology and evolution. PloS one, 13(7), e0200303.

Gehlbach, H., & Robinson, C. D. (2018). Mitigating illusory results through preregistration in education. Journal of Research on Educational Effectiveness, 11(2), 296-315.

Gelman, A. (2013). Preregistration of studies and mock reports. Political Analysis, 21(1), 40-41.

Goldin-Meadow, S. (2016). Preregistration, replication, and nonexperimental studies. APS Observer, 29(8).

Gonzales, J. E., & Cunningham, C. A. (2015). The promise of pre-registration in psychological research. Psychological Science Agenda.

Haven, L. T., & Van Grootel, L. (2019). Preregistering qualitative research. Accountability in Research, 26(3), 229-244.

Havron, N., Bergmann, C., & Tsuji, S. (2019, December 12). Preregistration in infant research – a primer. PsyArXiv.

Ikeda, A., Xu, H., Fuji, N., Zhu, S., & Yamada, Y. (2019). Questionable research practices following pre-registration. Japanese Psychological Review, 62(3), 281-295.

Johnson, A. H., & Cook, B. G. (2019). Preregistration in single-case design research. Exceptional Children, 86(1), 95-112.

Jonas, K. J., & Cesario, J. (2016). How can preregistration contribute to research in our field?. Comprehensive Results in Social Psychology 1(1-3).

Kern, F. G., & Gleditsch, K. S. (2017). Exploring pre-registration and pre-analysis plans for qualitative inference. Preprint ahead of publication, 1-15.

Kirtley, O., Lafit, G., Achterhof, R., Hiekkaranta, A., & Germeys, I. (2020). Making the black box transparent: A template and tutorial for (pre-) registration of studies using Experience Sampling Methods (ESM). Advances in Methods and Practices in Psychological Science.

Krypotos, A. M., Klugkist, I., Mertens, G., & Engelhard, I. M. (2019). A step-by-step guide on preregistration and effective data sharing for psychopathology research. Journal of Abnormal Psychology, 128(6), 517.

Kupferschmidt, K. (2018). More and more scientists are preregistering their studies. Should you? Science.

Lakens, D. (2019). The value of preregistration for psychological science: A conceptual analysis. Japanese Psychological Review, 62(3), 221-230.

Lee, H., Lamb, S. E., Bagg, M. K., Toomey, E., Cashin, A. G., & Moseley, G. L. (2018). Reproducible and replicable pain research: a critical review. Pain, 159(9), 1683-1689.

Lindsay, D. S., Simons, D. J., & Lilienfeld, S. O. (2016). Research preregistration 101. APS Observer, 29(10).

Mellor, D., Soderberg, C., & DeHaven, A. (2017). Preregistration on the OSF [Webinar]. Center for Open Science.

Mellor, D. T., & Nosek, B. A. (2018). Easy preregistration will benefit any research. Nature Human Behaviour, 2(2), 98-98.

Mellor, D. (2019). Looking Back on the Prereg Challenge and Forward To More Credible Research. Center for Open Science.

Mertens, G., & Krypotos, A. M. (2019). Preregistration of analyses of preexisting data. Psychologica Belgica, 59(1).

McCook, A. (2015).Did a clinical trial proceed as planned? New project finds out. Retraction Watch.

Miguel, E., Camerer, C., Casey, K., Cohen, J., Esterling, K.M., Gerber, A., …Van der Laan, M. (2014). Promoting transparency in social science research. Science, 343(6166), 30-31.

Monogan, J. (2014). The controversy of preregistration in social research. Berkeley Initiative for Transparency in the Social Sciences.

Monogan, J. (2015). Research preregistration in political science: The case, counterarguments, and a response to critiques. PS: Political Science & Politics, 48(3), 425-429.

Moore, D. A. (2016). Preregister if you want to. American Psychologist, 71(3), 238.

Moreau, D. (2019). Preregistration in the Context of Expertise Research: Benefits, Challenges, and Recommendations. Journal of Expertise/December, 2(4).

Nelson, L. (2014). Preregistration: not just for the empiro-zealots. Data Colada.

Neuroskeptic. (2014). Preregistration for data science? Discover.

NLM and NIH Clinical Trail Registry Site.

Nosek, B.A., Alter, G., Banks, G.C., Borsboom, D., Bowman, S.D., Breckler, S.J., … Yarkoni, T. (2015) Promoting an open research culture. Science, 348(6242), 1422-1425.

Nosek, B. A., Ebersole, C. R., DeHaven, A. C., & Mellor, D. T. (2018). The preregistration revolution. Proceedings of the National Academy of Sciences, 115(11), 2600-2606.

Nosek, B. A., & Lindsay, D. S. (2018). Preregistration becoming the norm in psychological science. APS Observer, 31(3)

Nosek, B. A., Beck, E. D., Campbell, L., Flake, J. K., Hardwicke, T. E., Mellor, D. T., … & Vazire, S. (2019). Preregistration is hard, and worthwhile. Trends in cognitive sciences, 23(10), 815-818.

Observer. (2020). Easier Done Than Said: Lessons From 6 Years of Preregistration. Association for Psychological Science.

Open practice badges. Association for psychological science.

Open science: APS and it’s journals.

Open Science Framework.

Orland, B. (2018). Preregistration: Improve Research Rigor, Reduce Bias [Webinar]. Center for Open Science.

Ozler, B. & Nosek, B. A. (2015). Preregistration of studies to avoid fishing and allow transparent discovery. Development Impact.

Parker, T., Fraser, H., Nakagawa, S., Hampton, J. O., Warburton, B., & Sandøe, P. (2019). 747 Making conservation science more reliable with preregistration and registered reports. Conservation Biology, 33(4).

Preregistration. Center for Open Science.

Preregistration at PLOS Biology and Call for COVID-19, SARS-CoV-2 Research. (2020). PLoS Blogs.

Preregistration of research plans. Psychological Science.

Registrations: Center for Open Science.

Registration Forms and Templates. Open Science Framework.

Rice, D. B., & Moher, D. (2019). Curtailing the Use of Preregistration: A Misused Term. Perspectives on Psychological Science, 14(6), 1105-1108.

Roettger, T. B. (2019). Preregistration in linguistic research: Applications, challenges, and limitations. PsyArXiv.

Sullivan, I., DeHaven, A., & Mellor, D. (2019). Open and reproducible research on open science framework. Current Protocols Essential Laboratory Techniques, 18(1), e32.

Szollosi, A., Kellen, D., Navarro, D. J., Shiffrin, R., van Rooij, I., Van Zandt, T., & Donkin, C. (2019). Is preregistration worthwhile. Trends in Cognitive Sciences, 24(2), 94-95.

Tackett, J. L., Brandes, C. M., & Reardon, K. W. (2019). Leveraging the Open Science Framework in clinical psychological assessment research. Psychological assessment, 31(12), 1386.

The Conversation. (2019). Pre-Registration, Data Sharing, And Registered Reports Are How The Science Community Can Reduce Shoddy Research. Science 2.0. 

Trust in science would be improved by study preregistration (2013). The Guardian.

The World Health Organization Registry: International Clinical Trails Registry Platform.

Van’t Veer, A. E., & Giner-Sorolla, R. (2016). Pre-registration in social psychology—A discussion and suggested template. Journal of Experimental Social Psychology, 67, 2-12.

Verschuere, B., Yasrebi-de Kom, F. M., van Zelm, I., & Lilienfeld, S. O. (2019). A plea for preregistration in personality disorders research: The case of psychopathy. Journal of Personality Disorders, 1-16.

Wagenmakers, E. J., Wetzels, R., Borsboom, D., van der Maas, H. L., & Kievit, R. A. (2012). An agenda for purely confirmatory research. Perspectives on Psychological Science, 7(6), 632-638.

Wagenmakers, E. & Dutilh, G. (2016). Seven Selfish Reasons for Preregistration. Association for Psychological Science.

Yamada, Y. (2018). How to crack pre-registration: Toward transparent and open science. Frontiers in psychology, 9, 1831.