Replication
The repetition of the methods and protocols employed in a study to determine whether the initial findings are robust and/or generalizable.
Allison, D. B., Shiffrin, R. M., & Stodden V. (2018) Reproducibility of research: Issues and proposed remedies. PNAS, 115(11), 2561-2562.
Alogna, V. K., Attaya, M. K., Aucoin, P., Bahník, Š., Birch, S., Bornstein, B., … & McCoy, S. (2014). Contribution to Alonga et al (2014). Registered replication report: Schooler & Engstler-Schooler (1990). Perspectives on Psychological Science, 9(5), 556-578.
Anderson, S. F., & Maxwell, S. E. (2015). There’s more than one way to conduct a replication study: Beyond statistical significance. Psychology Methodologies, 21(1), 1-12.
Ang, R. P. (1998). Use of the jacknife statistic to evaluate result replicability. The Journal of General Psychology, 125(3), 218-228.
Asendorpf, J. B., Conner, M., De Fruyt, F., De Houwer, J., Denissen, J. J., Fiedler, K., … & Wicherts, J. M. (2013). Recommendations for increasing replicability in psychology. European Journal of Personality, 27(2), 108-119.
Bakker, M., & Wicherts, J. M. (2011). The (mis) reporting of statistical results in psychology journals. Behavior Research Methods, 43(3), 666-678.
Begley, C. G. (2013). Reproducibility: six red flags for suspect work. Nature,497(7450), 433-434.
Begley, C. G., & Ioannidis, J. P. (2015). Reproducibility in Science Improving the Standard for Basic and Preclinical Research. Circulation Research, 116(1), 116-126.
Brandt, M.J., IJzerman, H., Dijksterhius, A., Farach, F.J., Geller, J., Giner-Sorolla, R., … van ‘t Veer, A. (2014). The replication recipe: What makes for a convincing replication? Journal of Experimental Social Psychology, 50, 217-224.
Brodeur, A., Lé, M., Sangnier, M., & Zylberberg, Y. (2012). Star wars: The empirics strike back. Paris School of Economics Working Paper, (2012-29).
Bryan, C. J., Yeager, D. S., & O’Brien, J.,M. (2019). Replicator degrees of freedom allow publication of misleading failures to replicate. Proceedings of the National Academy of Sciences of the United States of America, 116(51), 25535.
Button, K. S., Ioannidis, J. P., Mokrysz, C., Nosek, B. A., Flint, J., Robinson, E. S., & Munafò, M. R. (2013). Empirical evidence for low reproducibility indicates low pre-study odds. Nature Reviews Neuroscience, 14(12), 877-877.
Button, K. S., Ioannidis, J. P., Mokrysz, C., Nosek, B. A., Flint, J., Robinson, E. S., & Munafò, M. R. (2013). Power failure: why small sample size undermines the reliability of neuroscience. Nature Reviews Neuroscience, 14(5), 365-376.
Camerer, C., Dreber, A., Holzmeister, F., et al. (2018). Evaluating the replicability of social science experiments in Nature and Science between 2010 and 2015. Nature Human Behavior, 2, 637-644.
Cesario, J. (2014). Priming, replication, and the hardest science. Perspectives on Psychological Science, 9(1), 40-48.
Coffman, L. C., & Niederle, M. (2015). Pre-analysis Plans Have Limited Upside, Especially Where Replications Are Feasible. Journal of Economic Perspectives, 29(3): 81-98.
Collins, F. S., & Tabak, L. A. (2014). Policy: NIH plans to enhance reproducibility. Nature, 505(7485), 612-613.
Dijksterhuis, A. (2014). Welcome Back Theory! Perspectives on Psychological Science, 9(1), 72-75.
Donoho, D. L., & Huo, X. (2004). Beamlab and reproducible research. International Journal of Wavelets, Multiresolution and Information Processing,2(04), 391-414.
Doyen, S., Klein, O., Pichon, C., & Cleeremans, A. (2012). Behavioral priming: It’s all in the mind, but whose mind? PloS One, 7(1), e29081.
Duncan, L. E., & Keller, M. C. (2011). A critical review of the first 10 years of candidate gene-by-environment interaction research in psychiatry. Perspectives, 168(10).
Eubank, N. (2016). Lessons from a decade of replications at the quarterly journal of political science.PS, Political Science & Politics, 49(2), 273-276.
Fanelli, D., & Ioannidis, J. P. (2013). US studies may overestimate effect sizes in softer research. Proceedings of the National Academy of Sciences, 110(37), 15031-15036.
Francis, G. (2012). Publication bias and the failure of replication in experimental psychology. Psychonomic Bulletin & Review, 19(6), 975-91.
Francis, G. (2012). The psychology of replication and replication in psychology. Perspectives on Psychological Science, 7(6), 585-594.
Frank, M. C., & Saxe, R. (2012). Teaching replication. Perspectives on Psychological Science, 7(6), 600-604.
Friederike, H., Dorothe, K., & Rainer, B. (2020). Replication crisis = trust crisis? the effect of successful vs failed replications on laypeople’s trust in researchers and research. Public Understanding of Science, 29(3), 270-288.
Galak, J., LeBoeuf, R. A., Nelson, L. D., & Simmons, J. P. (2012). Correcting the past: Failures to replicate psi. Journal of Personality and Social Psychology,103(6), 933.
Galak, J., & Meyvis, T. (2012). You Could Have Just Asked Reply to Francis (2012). Perspectives on Psychological Science, 7(6), 595-596.
Gelman, A. (2018). Don’t characterize replications as successes or failures. Behavioral and Brain Sciences, 41.
Gilbert, D. T., King, G., Pettigrew, S., & Wilson, T. D. (2016). Comment on “Estimating the reproducibility of psychological science”. Science, 351(6277), 1037.
Grahe, J. E., Reifman, A., Hermann, A. D., Walker, M., Oleson, K. C., Nario-Redmond, M., & Wiebe, R. P. (2012). Harnessing the undiscovered resource of student research projects. Perspectives on Psychological Science, 7(6), 605-607.
Hamermesh, D. S. (2007). Viewpoint: Replication in economics. Canadian Journal of Economics/Revue canadienne d’économique, 40(3), 715-733.
Harris, C. R., Coburn, N., Rohrer, D., & Pashler, H. (2013). Two failures to replicate high-performance-goal priming effects. PloS one, 8(8), e72467.
Heller, R., Bogomolov, M., & Benjamini, Y. (2014). Deciding whether follow-up studies have replicated findings in a preliminary large-scale omics study. Proceedings of the National Academy of Sciences, 111(46), 16262-16267.
Holden, G., Barker, K., Kuppens, S., Rosenberg, G., & LeBreton, J. (2015). A replication of failure, not a failure to replicate. Research on Social Work Practice, 25(3), 313.
Hothorn, T., & Leisch, F. (2011). Case studies in reproducibility. Briefings in Bioinformatics, 12(3), 288-300.
Ioannidis, J. P. (2006). Non-replication and inconsistency in the genome-wide association setting. Human Heredity, 64(4), 203-213.
Ioannidis, J. P. (2013). To Replicate or Not to Replicate: The Case of Pharmacogenetic Studies Have Pharmacogenomics Failed, or Do They Just Need Larger-Scale Evidence and More Replication?. Circulation: Cardiovascular Genetics, 6(4), 413-418.
Ioannidis, J. P. (2012). Scientific inbreeding and same-team replication: type D personality as an example. Journal of Psychosomatic Research, 73(6), 408-410.
Ioannidis, J. P., Nosek, B., & Iorns, E. (2012). Reproducibility concerns. Nature Medicine, 18(12), 1736-1737.
Ioannidis, J. P., Ntzani, E. E., Trikalinos, T. A., & Contopoulos-Ioannidis, D. G. (2001). Replication validity of genetic association studies. Nature Genetics,29(3), 306-309.
Iorns, E., Gunn, W., Erath, J., Rodriguez, A., Zhou, J., Benzinou, M., & Reproducibility Initiative. (2014). Replication Attempt:“Effect of BMAP-28 Antimicrobial Peptides on Leishmania Major Promastigote and Amastigote Growth: Role of Leishmanolysin in Parasite Survival”. PloS One, 9(12), e114614.
Ishiyama, J. (2014). Replication, research transparency, and journal publications: Individualism, community models, and the future of replication studies. PS, Political Science & Politics, 47(1),78-83.
Jager, L. R., & Leek, J. T. (2014). An estimate of the science-wise false discovery rate and application to the top medical literature. Biostatistics, 15(1), 1-12.
Jasny, B. R., Chin, G., Chong, L., & Vignieri, S. (2011). Again, and again, and again…. Science, 334(6060), 1225-1225.
Johnson, D. J., Cheung, F., & Donnellan, M. B. (2014). Does cleanliness influence moral judgments? A direct replication of Schnall, Benton, and Harvey (2008). Social Psychology, 45(3), 209.
King, G. (1995). Replication, replication. PS: Political Science & Politics, 28(03), 444-452.
Klein, R.A., Ratliff, K.A., Vianello, M., Adams Jr., R.B., Bahnik, S., Bernstein, M.J., et al. (2014). Investigating variation in replicability: A “many labs” replication project. Social Psychology, 45, 142-152.
Klein, S. B. (2014). What can recent replication failures tell us about the theoretical commitments of psychology? Theory & Psychology, 24(3), 326.
Kunert, R. (2016). Internal conceptual replications do not increase independent replication success. Psychonomic Bulletin & Review, 23(5), 1631-1638.
Laine, C., Guallar, E., Mulrow, C., Taichman, D. B., Cornell, J. E., Cotton, D., … & Goodman, S. N. (2013). Closing in on the truth about recombinant human bone morphogenetic protein-2: Evidence synthesis, data sharing, peer review, and reproducible research. Annals of Internal Medicine, 158(12), 916-918.
Leggett, N. C., Thomas, N. A., Loetscher, T., & Nicholls, M. E. (2013). The life of p:“Just significant” results are on the rise. The Quarterly Journal of Experimental Psychology, 66(12), 2303-2309.
Li, Q., Brown, J. B., Huang, H., & Bickel, P. J. (2011). Measuring reproducibility of high-throughput experiments. The Annals of Applied Statistics, 5(3), 1752-1779.
Kuehberger, A., & Schulte-Mecklenbeck, M. (2018). Selecting target papers for replication. Behavioraland Brain Sciences, 41.
Koole, S. L., & Lakens, D. (2012). Rewarding replications a sure and simple way to improve psychological science. Perspectives on Psychological Science, 7(6), 608-614.
Madden, C. S., Easley, R. W., & Dunn, M. G. (1995). How journal editors view replication research. Journal of Advertising, 24(4), 77-87.
Makel, M. C., & Plucker, J. A. (2014). Facts Are More Important Than Novelty Replication in the Education Sciences. Educational Researcher, 0013189X14545513.
Makel, M. C., Plucker, J. A., & Hegarty, B. (2012). Replications in Psychology Research How Often Do They Really Occur?. Perspectives on Psychological Science, 7(6), 537-542.
Masicampo, E. J., & Lalande, D. R. (2012). A peculiar prevalence of p values just below. 05. The Quarterly Journal of Experimental Psychology, 65(11), 2271-2279.
Maxwell, S. E., Lau, M. Y., & Howard, G. S. (2015). Is psychology suffering from a replication crisis? what does “failure to replicate” really mean? The American Psychologist, 70(6), 487.
Molina, H., Parmigiani, G., & Pandey, A. (2005). Assessing reproducibility of a protein dynamics study using in vivo labeling and liquid chromatography tandem mass spectrometry. Analytical Chemistry, 77(9), 2739-2744.
Nieuwenstein, M., & Rijn, H. v. (2012). The unconscious thought advantage: Further replication failures from a search for confirmatory evidence. Judgment and Decision Making, 7(6), 779.
Neuliep, J. W., & Crandall, R. (1993). Reviewer bias against replication research. Journal of Social Behavior & Personality.
Open Science Collaboration. (2015). Estimating the reproducibility of psychological science. Science, 349(6251), aac4716.
Pashler, H., Coburn, N., & Harris, C. R. (2012). Priming of social distance? Failure to replicate effects on social and food judgments. PloS One, 7(8), e42510.
Pashler, H., & Harris, C. R. (2012). Is the replicability crisis overblown? Three arguments examined. Perspectives on Psychological Science, 7(6), 531-536.
Pashler, H., & Wagenmakers, E. J. (2012). Editors’ Introduction to the Special Section on Replicability in Psychological Science A Crisis of Confidence?. Perspectives on Psychological Science, 7(6), 528-530.
Patil, P., & Parmigiani, G. (2018) Training replicable predictors in multiple studies. PNAS, 115(11), 2578-2583.
Peers, I. S., Ceuppens, P. R., & Harbron, C. (2012). In search of preclinical robustness. Nature Reviews Drug Discovery, 11(10), 733-734.
Pereira, T. V., & Ioannidis, J. P. (2011). Statistically significant meta-analyses of clinical trials have modest credibility and inflated effects. Journal of Clinical Epidemiology, 64(10), 1060-1069.
Poldrack, R. A., & Poline, J. B. (2014). The publication and reproducibility challenges of shared data. Trends in Cognitive Sciences.
Prinz, F., Schlange, T., & Asadullah, K. (2011). Believe it or not: how much can we rely on published data on potential drug targets? Nature Reviews Drug Discovery, 10(9), 712-712.
Roediger, H. L. (2012). Psychology’s woes and a partial cure: The value of replication. APS Observer, 25(2), 9.
Rohrer, D., Pashler, H., & Harris, C. R. (2015). Do subtle reminders of money change people’s political views? Journal of Experimental Psychology: General, 144(4), e73–e85.
Rünger, D., Schwager, S., & Frensch, P. A. (2010). Across-task conflict regulation: A replication failure. Journal of Experimental Psychology, 36(1), 136.
Schimmack, U. (2018). The replicability revolution. Behavioral and Brain Sciences, 41.
Schmidt, S. (2009). Shall we really do it again? The powerful concept of replication is neglected in the social sciences. Review of General Psychology,13(2), 90.
Seyedsayamdost, H. (2015). On gender and philosophical intuition: Failure of replication and other negative results. Philosophical Psychology, 28(5), 642.
Simons, D. J. (2014). The value of direct replication. Perspectives on Psychological Science, 9(1), 76-80.
Simons, D. J., Holcombe, A. O., & Spellman, B. A. (2014). An Introduction to Registered Replication Reports at Perspectives on Psychological Science. Perspectives on Psychological Science, 9(5), 552-555.
Simonsohn, U., Nelson, L. D., & Simmons, J. P. (2014). p-Curve and Effect Size Correcting for Publication Bias Using Only Significant Results. Perspectives on Psychological Science, 9(6), 666-681.
Simmons, J. P., Nelson, L. D., & Simonsohn, U. (2011). False-positive psychology undisclosed flexibility in data collection and analysis allows presenting anything as significant. Psychological Science, 0956797611417632.
Siontis, G. C., Tzoulaki, I., Castaldi, P. J., & Ioannidis, J. P. (2015). External validation of new risk prediction models is infrequent and reveals worse prognostic discrimination. Journal of Clinical Epidemiology, 68(1), 25-34.
Spellman, B. A., & Kahneman, D. (2018). What the replication reformation wrought. Behavioral and Brain Sciences, 41.
Stanford, R. G. (2003). Research strategies for enhancing conceptual development and replicability. The Journal of Parapsychology, 67(1), 17-51.
Stodden, V. (2009). Enabling reproducible research: Open licensing for scientific innovation. International Journal of Communications Law and Policy, Forthcoming.
Stodden, V. (2009). The legal framework for reproducible scientific research: Licensing and copyright. Computing in Science & Engineering, 11(1), 35-40.
Stodden, V., Borwein, J., & Bailey, D. H. (2013). Setting the default to reproducible. Computational Science Research. SIAM News, 46, 4-6.
Stodden, V., Leisch, F., & Peng, R. D. (Eds.). (2014). Implementing Reproducible Research. CRC Press.
Stroebe, W., & Strack, F. (2014). The alleged crisis and the illusion of exact replication. Perspectives on Psychological Science, 9(1), 59-71.
Vandewalle, P., Kovacevic, J., & Vetterli, M. (2009). Reproducible research in signal processing. Signal Processing Magazine, IEEE, 26(3), 37-47.
Waldron, L., Haibe-Kains, B., Culhane, A. C., Riester, M., Ding, J., Wang, X. V., … & Parmigiani, G. (2014). Comparative meta-analysis of prognostic gene signatures for late-stage ovarian cancer. Journal of the National Cancer Institute, 106(5), dju049.
Wicherts, J.M., Bakker, M., & Molenaar, D. (2011). Willingness to share research data is related to the strength of the evidence and the quality of reporting of statistical results. PLoS One, 6, e26828.
Yong, E. (2012). In the wake of high profile controversies, psychologists are facing up to problems with replication. Nature, 483, 298-300.
Zwaan, R. A., Etz, A., Lucas, R. E., & Donnellan, M. B. (2018). Making replication mainstream. Behavioral and Brain Sciences, 41.