Academic Publications



Methodological Critiques

investigations into the reliability and validity of the design, implementation and analysis of research protocols.

Bakker, M., van Dijk, A., & Wicherts, J.M. (2012). The rules of the game called psychological science.  Perspectives on Psychological Science, 7, 543-554.

Begley, C. G., & Ellis, L. M. (2012). Drug development: Raise standards for preclinical cancer research. Nature, 483(7391), 531-533.

Blanton, H., & Jaccard, J. (2006). Arbitrary metrics in psychology. American Psychologist, 61(1), 27.

Castaldi, P. J., Dahabreh, I. J., & Ioannidis, J. P. (2011). An empirical assessment of validation practices for molecular classifiers. Briefings in Bioinformatics, 12(3), 189-202.

Clayton, J. A., & Collins, F. S. (2014). NIH to balance sex in cell and animal studies. Nature, 509(7500), 282-283.

Collins, G. S., Reitsma, J. B., Altman, D. G., & Moons, K. G. (2015). Transparent reporting of a multivariable prediction model for individual prognosis or diagnosis (TRIPOD): the TRIPOD Statement. BMC Medicine,13(1), 1.

Cornell, J. E., Mulrow, C. D., Localio, R., Stack, C. B., Meibohm, A. R., Guallar, E., & Goodman, S. N. (2014). Random-effects meta-analysis of inconsistent effects: a time for change. Annals of Internal Medicine, 160(4), 267-270.

Cyranoski, D. (2006). Verdict: Hwang’s human stem cells were all fakes. Nature, 439(7073), 122-123.

Davidson, A. J., & Carlin, J. B. (2008). What a reviewer wants. Pediatric Anesthesia, 18(12), 1149-1156.

Doshi, P., Goodman, S. N., & Ioannidis, J. P. (2013). Raw data from clinical trials: within reach?. Trends in Pharmacological Sciences, 34(12), 645-647.

Eubanks, P., & Schaeffer, J. D. (2008). A Kind Word for Bullshit: The Problem of Academic Writing. The National Council of Teachers of English, 372-388.

Fanelli D (2010) “Positive” Results Increase Down the Hierarchy of the Sciences. PLoS One 5(4): e10068. doi:10.1371/journal.pone.0010068

Fanelli, D. (2013). Positive results receive more citations, but only in some disciplines. Scientometrics, 94(2), 701-709.

Fanelli, D. (2012). Negative results are disappearing from most disciplines and countries. Scientometrics, 90(3), 891-904.

Fang, F. C., Steen, R. G., & Casadevall, A. (2012). Misconduct accounts for the majority of retracted scientific publications. Proceedings of the National Academy of Sciences, 109(42), 17028-17033.

Fowler, J. H., & Christakis, N. A. (2008). Estimating peer effects on health in social networks: A response to Cohen-Cole and Fletcher; and Trogdon, Nonnemaker, and Pais. Journal of Health Economics, 27(5), 1400-1405.

Francis, G. (2014). The frequency of excess success for articles in Psychological Science. Psychonomic Bulletin & Review, 21(5), 1180-1187.

Friggeri, A., Adamic, L. A., Eckles, D., & Cheng, J. (2014). Rumor Cascades. 101-110.

Gardner, W., Lidz, C. W., & Hartwig, K. C. (2005). Authors’ reports about research integrity problems in clinical trials. Contemporary Clinical Trials, 26(2), 244-251.

Goodman, S. N. (2013). Bayesian methods for evidence evaluation: are we there yet?. Circulation, CIRCULATIONAHA-113.

Grove, W. M., & Meehl, P. E. (1996). Comparative efficiency of informal (subjective, impressionistic) and formal (mechanical, algorithmic) prediction procedures: The clinical–statistical controversy. Psychology, Public Policy, and Law, 2(2), 293.

Habbema, J. D. F., Wilt, T. J., Etzioni, R., Nelson, H. D., Schechter, C. B., Lawrence, W. F., … & Feuer, E. J. (2014). Models in the development of clinical practice guidelines. Annals of Internal Medicine, 161(11), 812-818.

Hayden, J. E. (1999). Digital manipulation in scientific images: some ethical considerations. The Journal of Biocommunication, 27(1), 11-19.

Ioannidis, J. P. (2001). Clinical trials: what a waste. J Clin Epidemiol, 54, 877-83.

Ioannidis, J. P. (2014). Discussion: Why “An estimate of the science-wise false discovery rate and application to the top medical literature” is false. Biostatistics, 15(1), 28-36.

Ioannidis, J. P. (2014). Modeling and Research on Research. Clinical Chemistry, 60(9), 1238-1239

Ioannidis, J. P. (2008, July). Perfect study, poor evidence: interpretation of biases preceding study design. In Seminars in Hematology (Vol. 45, No. 3, pp. 160-166). WB Saunders.

Ioannidis, J. P. (2014). Research accomplishments that are too good to be true. Intensive Care Medicine, 40(1), 99-101.

Ioannidis, J. P. (2008). Some main problems eroding the credibility and relevance of randomized trials. Bulletin of the NYU Hospital for Joint Diseases,66(2), 135-139.

Ioannidis, J. P., Horbar, J. D., Ovelman, C. M., Brosseau, Y., Thorlund, K., Buus-Frank, M. E., … & Soll, R. F. (2015). Completeness of main outcomes across randomized trials in entire discipline: survey of chronic lung disease outcomes in preterm infants. BMJ, 350, h72.

Karcz, M., & Papadakos, P. J. (2011). The Consequences of fraud and deceit in medical research. Canadian Journal of Respiratory Therapy, 47(1), 18-27.

Kerr, N.L. (1998).  HARKing: Hypothesizing After the Results Are Known. Personality and Social Psychology Review, 2(3), 196-217.

Kornfeld, D. S. (2012). Perspective: Research misconduct: The search for a remedy. Academic Medicine, 87(7), 877-882.

Korpela, K. M. (2010). How long does it take for the scientific literature to purge itself of fraudulent material?: the Breuning case revisited. Current Medical Research & Opinion, 26(4), 843-847.

Kumar, M. N. (2008). A review of the types of scientific misconduct in biomedical research. Journal of Academic Ethics, 6(3), 211-228.

Lau, J., Ioannidis, J. P., & Schmid, C. H. (1998). Summing up evidence: one answer is not always enough. The Lancet, 351(9096), 123-127.

Long, T. C., Errami, M., George, A. C., Sun, Z., & Garner, H. R. (2009). Responding to possible plagiarism. < target=”_blank”i>Science, 323(5919), 1293-1294.

Lord, C. G., Ross, L., & Lepper, M. R. (1979). Biased assimilation and attitude polarization: The effects of prior theories on subsequently considered evidence. Journal of Personality and Social Psychology, 37(11), 2098.

May, C., Campbell, S., & Doyle, H. (1998). Research misconduct: A pilot study of British addiction researchers. Addiction Research & Theory, 6(4), 371-373.

Meehl, P.A. (1990). Appraising and amending theories: The strategy of Lakatosian defense and two principles that warrant using it. Psychological Inquiry, 1(2), 108-141.

Meehl, P. E. (1959). Some ruminations on the validation of clinical procedures. Canadian Journal of Psychology/Revue Canadienne de Psychologie, 13(2), 102.

Meehl, P. E. (1959). Structured and projective tests: Some common problems in validation. Journal of Projective Techniques, 23(3), 268-272.

Meehl, P.A. (1967). Theory-testing in psychology and physics: A methodological paradox. Philosophy of Science, 34, 103-115.

Meehl, P. E. (1990). Why summaries of research on psychological theories are often uninterpretable. Psychological Reports, 66(1), 195-244.

Meehl, P. E., & Waller, N. G. (2002). The Path Analysis Controversy: A new statistical approach to strong appraisal of verisimilitude. Psychological Methods, 7(3), 283.

Michell, J. (1997). Quantitative science and the definition of measurement in psychology. British Journal of Psychology, 88(3), 355-383.

Miller, A. G., McHoskey, J. W., Bane, C. M., & Dowd, T. G. (1993). The attitude polarization phenomenon: Role of response measure, attitude extremity, and behavioral consequences of reported attitude change. Journal of Personality and Social Psychology, 64(4), 561.

Mills, E. J., Kanters, S., Thorlund, K., Chaimani, A., Veroniki, A. A., & Ioannidis, J. P. (2013). The effects of excluding treatments from network meta-analyses: survey. BMJ, 347.

Mumford, J. A., Davis, T., & Poldrack, R. A. (2014). The impact of study design on pattern estimation for single-trial multivariate pattern analysis. NeuroImage,103, 130-138.

Murad, M. H., Montori, V. M., Ioannidis, J. P., Jaeschke, R., Devereaux, P. J., Prasad, K., … & Guyatt, G. (2014). How to read a systematic review and meta-analysis and apply the results to patient care: users’ guides to the medical literature. JAMA, 312(2), 171-179.

Patel, C. J., & Ioannidis, J. P. (2014). Studying the elusive environment in large scale. JAMA, 311(21), 2173-2174.

Patsopoulos, N. A., & Ioannidis, J. P. (2009). The use of older studies in meta-analyses of medical interventions: a survey. Open Medicine, 3(2), e62.

Pennycook, G., Cheyne, J. A., Barr, N., Koehler, D. J., & Fugelsang, J. A. (2015). On the reception and detection of pseudo-profound bullshit. Judgement and Decision Making, 10(6), 549-563.

Plunk, V. (2013). Who’s afraid of peer review?. Science, 342, 60-65.

Prasad, V., & Ioannidis, J. P. (2014). Evidence-based de-implementation for contradicted, unproven, and aspiring healthcare practices. Implement Sci, 9(1), 5908-9.

Robinson, K. A., & Goodman, S. N. (2011). A systematic examination of the citation of prior research in reports of randomized, controlled trials. Annals of Internal Medicine, 154(1), 50-55.

Schooler, J. (2011). Unpublished results hide the decline effect. Nature470(7335), 437.

Schuemie, M. J., Ryan, P. B., Suchard, M. A., Shahn, Z., & Madigan, D. (2014). Discussion: An estimate of the science-wise false discovery rate and application to the top medical literature. Biostatistics15(1), 36-39.

Simmons, J.P., Nelson, L.D., & Simonsohn, U. (2011). False-positive psychology: Undisclosed flexibility in data collection and analysis allows presenting anything as significant. Psychological Science, 22, 1359-1366.

Stack, C. B., Localio, A. R., Griswold, M. E., Goodman, S. N., & Mulrow, C. D. (2013). Handling of rescue and missing data affects synthesis and interpretation of evidence: the sodium–glucose cotransporter 2 inhibitor example. Annals of Internal Medicine, 159(4), 285-288.

Sun, X., Ioannidis, J. P., Agoritsas, T., Alba, A. C., & Guyatt, G. (2014). How to use a subgroup analysis: users’ guide to the medical literature. JAMA, 311(4), 405-411.

Tavare, A. (2011). Managing research misconduct: is anyone getting it right?. BMJ, 343.

ter Riet, G., Chesley, P., Gross, A. G., Siebeling, L., Muggensturm, P., Heller, N., … & Puhan, M. A. (2013). All That Glitters Isn’t Gold: A Survey on Acknowledgment of Limitations in Biomedical Studies. PloS One, 8(11), e73623.

Tian, L., Alizadeh, A. A., Gentles, A. J., & Tibshirani, R. (2014). A Simple Method for Estimating Interactions Between a Treatment and a Large Number of Covariates. Journal of the American Statistical Association, 00-00.

Trikalinos, N. A., Evangelou, E., & Ioannidis, J. P. (2008). Falsified papers in high-impact journals were slow to retract and indistinguishable from nonfraudulent papers. Journal of Clinical Epidemiology, 61(5), 464-470.

Trinquart, L., Ioannidis, J. P., Chatellier, G., & Ravaud, P. (2014). A test for reporting bias in trial networks: simulation and case studies. BMC Medical Research Methodology, 14(1), 112.

Tzoulaki, I., Ebbels, T. M., Valdes, A., Elliott, P., & Ioannidis, J. P. (2014). Design and analysis of metabolomics studies in epidemiologic research: a primer on-omic technologies. American Journal of Epidemiology, 180(2), 129-139.

Tzoulaki, I., Seretis, A., Ntzani, E. E., & Ioannidis, J. P. (2014). Mapping the expanded often inappropriate use of the Framingham Risk Score in the medical literature. Journal of Clinical Epidemiology, 67(5), 571-577.

Wagenmakers, E.J., Wetzels, R., Borsboom, D., van der Maas, H.L.J., & Kievit, R.A. (2012). An agenda for purely confirmatory research.  Perspectives on Psychological Science, 7, 632-638.

Waller, N. G., & Meehl, P. E. (2002). Risky tests, verisimilitude, and path analysis. Psychological Methods, Vol 7(3), Sep 2002, 323-337.

Wernerova, M., & Hudlicky, T. (2010). On the Practical Limits of Determining Isolated Product Yields and Ratios of Stereoisomers: Reflections, Analysis, and Redemption. Synlett, 2010(18), 2701-2707. 

Zintzaras, E., & Ioannidis, J. P. (2014). HELOW: A program for testing extreme homogeneity in meta-analysis. Computer Methods and Programs in Biomedicine, 117(2), 383-386.

BPS invites readers to send (to relevant papers and links to add to this website.