Methodological Critiques
Investigations into the reliability and validity of the design, implementation and analysis of research protocols.
Alexander, J. A., & Hearld, L. R. (2009). Review: What can we learn from quality improvement research?: A critical review of research methods: MCRR MCRR. Medical Care Research and Review, 66(3), 235.
Andersen, M. B., McCullagh, P., & Wilson, G. J. (2007). But what do the numbers really tell us?: Arbitrary metrics and effect size reporting in sport psychology research. Journal of Sport & Exercise Psychology, 29(5), 664.
Bakker, M., van Dijk, A., & Wicherts, J.M. (2012). The rules of the game called psychological science. Perspectives on Psychological Science, 7, 543-554.
Begley, C. G., & Ellis, L. M. (2012). Drug development: Raise standards for preclinical cancer research. Nature, 483(7391), 531-533.
Bergkvist, L., & Rossiter, J. R. (2007). The predictive validity of multiple-item versus single-item measures of the same constructs. Journal of Marketing Research, 44(2), 175-184.
Blanton, H., & Jaccard, J. (2006). Arbitrary metrics in psychology. American Psychologist, 61(1), 27.
Borsboom, D., Mellenbergh, G. J., & van Heerden, J. (2004). The concept of validity. Psychological Review, 111(4), 1061-1071.
Castaldi, P. J., Dahabreh, I. J., & Ioannidis, J. P. (2011). An empirical assessment of validation practices for molecular classifiers. Briefings in Bioinformatics, 12(3), 189-202.
Clayton, J. A., & Collins, F. S. (2014). NIH to balance sex in cell and animal studies. Nature, 509(7500), 282-283.
Collins, G. S., Reitsma, J. B., Altman, D. G., & Moons, K. G. (2015). Transparent reporting of a multivariable prediction model for individual prognosis or diagnosis (TRIPOD): the TRIPOD Statement. BMC Medicine,13(1), 1.
Coltman, T., Devinney, T. M., Midgley, D. F., & Venaik, S. (2008). Formative versus reflective measurement models: Two applications of formative measurement. Journal of Business Research, 61(12), 1250-1262.
Cornell, J. E., Mulrow, C. D., Localio, R., Stack, C. B., Meibohm, A. R., Guallar, E., & Goodman, S. N. (2014). Random-effects meta-analysis of inconsistent effects: a time for change. Annals of Internal Medicine, 160(4), 267-270.
Cyranoski, D. (2006). Verdict: Hwang’s human stem cells were all fakes. Nature, 439(7073), 122-123.
Davidson, A. J., & Carlin, J. B. (2008). What a reviewer wants. Pediatric Anesthesia, 18(12), 1149-1156.
Deborah K van, d. H. (2003). Qualitative methods in social research / qualitative research methods. The Canadian Review of Sociology and Anthropology, 40(1), 116-118.
Doshi, P., Goodman, S. N., & Ioannidis, J. P. (2013). Raw data from clinical trials: within reach?. Trends in Pharmacological Sciences, 34(12), 645-647.
Embretson, S. E. (1998). A cognitive design system approach to generating valid tests: Application to abstract reasoning. Psychological Methods, 3(3), 380-396.
Embretson, S. E. (2006). The continued search for nonarbitrary metrics in psychology. The American Psychologist, 61(1), 50-55.
Enders, C. K., & Tofighi, D. (2007). Centering predictor variables in cross-sectional multilevel models: A new look at an old issue. Psychological Methods, 12(2), 121-138.
Eubanks, P., & Schaeffer, J. D. (2008). A Kind Word for Bullshit: The Problem of Academic Writing. The National Council of Teachers of English, 372-388.
Fanelli D (2010) “Positive” Results Increase Down the Hierarchy of the Sciences. PLoS One 5(4): e10068. doi:10.1371/journal.pone.0010068
Fanelli, D. (2013). Positive results receive more citations, but only in some disciplines. Scientometrics, 94(2), 701-709.
Fanelli, D. (2012). Negative results are disappearing from most disciplines and countries. Scientometrics, 90(3), 891-904.
Fang, F. C., Steen, R. G., & Casadevall, A. (2012). Misconduct accounts for the majority of retracted scientific publications. Proceedings of the National Academy of Sciences, 109(42), 17028-17033.
Fowler, J. H., & Christakis, N. A. (2008). Estimating peer effects on health in social networks: A response to Cohen-Cole and Fletcher; and Trogdon, Nonnemaker, and Pais. Journal of Health Economics, 27(5), 1400-1405.
Francis, G. (2014). The frequency of excess success for articles in Psychological Science. Psychonomic Bulletin & Review, 21(5), 1180-1187.
Friggeri, A., Adamic, L. A., Eckles, D., & Cheng, J. (2014). Rumor Cascades. 101-110.
Gardner, W., Lidz, C. W., & Hartwig, K. C. (2005). Authors’ reports about research integrity problems in clinical trials. Contemporary Clinical Trials, 26(2), 244-251.
Goodman, S. N. (2013). Bayesian methods for evidence evaluation: are we there yet?. Circulation, CIRCULATIONAHA-113.
Gräbner, C., & Kapeller, J. (2015). New perspectives on institutionalist pattern modeling: Systemism, complexity, and agent-based modeling. Journal of Economic Issues, 49(2), 433-440.
Grove, W. M., & Meehl, P. E. (1996). Comparative efficiency of informal (subjective, impressionistic) and formal (mechanical, algorithmic) prediction procedures: The clinical–statistical controversy. Psychology, Public Policy, and Law, 2(2), 293.
Habbema, J. D. F., Wilt, T. J., Etzioni, R., Nelson, H. D., Schechter, C. B., Lawrence, W. F., … & Feuer, E. J. (2014). Models in the development of clinical practice guidelines. Annals of Internal Medicine, 161(11), 812-818.
Hayden, J. E. (1999). Digital manipulation in scientific images: some ethical considerations. The Journal of Biocommunication, 27(1), 11-19.
Hopper, K. (2008). Qualitative and quantitative research: Two cultures. Psychiatric Services, 59(7), 711.
Ioannidis, J. P. (2001). Clinical trials: what a waste. J Clin Epidemiol, 54, 877-83.
Ioannidis, J. P. (2014). Discussion: Why “An estimate of the science-wise false discovery rate and application to the top medical literature” is false. Biostatistics, 15(1), 28-36.
Ioannidis, J. P. (2014). Modeling and Research on Research. Clinical Chemistry, 60(9), 1238-1239
Ioannidis, J. P. (2008, July). Perfect study, poor evidence: interpretation of biases preceding study design. In Seminars in Hematology (Vol. 45, No. 3, pp. 160-166). WB Saunders.
Ioannidis, J. P. (2014). Research accomplishments that are too good to be true. Intensive Care Medicine, 40(1), 99-101.
Ioannidis, J. P. (2008). Some main problems eroding the credibility and relevance of randomized trials. Bulletin of the NYU Hospital for Joint Diseases,66(2), 135-139.
Ioannidis, J. P., Horbar, J. D., Ovelman, C. M., Brosseau, Y., Thorlund, K., Buus-Frank, M. E., … & Soll, R. F. (2015). Completeness of main outcomes across randomized trials in entire discipline: survey of chronic lung disease outcomes in preterm infants. BMJ, 350, h72.
Karcz, M., & Papadakos, P. J. (2011). The Consequences of fraud and deceit in medical research. Canadian Journal of Respiratory Therapy, 47(1), 18-27.
Kerr, N.L. (1998). HARKing: Hypothesizing After the Results Are Known. Personality and Social Psychology Review, 2(3), 196-217.
Knottnerus, J. A., & Tugwell, P. (2010). Credibility and methodology of effectiveness research keep us busy. Journal of Clinical Epidemiology, 63(2), 115-6.
Kornfeld, D. S. (2012). Perspective: Research misconduct: The search for a remedy. Academic Medicine, 87(7), 877-882.
Korpela, K. M. (2010). How long does it take for the scientific literature to purge itself of fraudulent material?: the Breuning case revisited. Current Medical Research & Opinion, 26(4), 843-847.
Kumar, M. N. (2008). A review of the types of scientific misconduct in biomedical research. Journal of Academic Ethics, 6(3), 211-228.
Lau, J., Ioannidis, J. P., & Schmid, C. H. (1998). Summing up evidence: one answer is not always enough. The Lancet, 351(9096), 123-127.
Long, T. C., Errami, M., George, A. C., Sun, Z., & Garner, H. R. (2009). Responding to possible plagiarism. < target=”_blank”i>Science, 323(5919), 1293-1294.
Lord, C. G., Ross, L., & Lepper, M. R. (1979). Biased assimilation and attitude polarization: The effects of prior theories on subsequently considered evidence. Journal of Personality and Social Psychology, 37(11), 2098.
Maggino, F., & Facioni, C. (2017). Erratum to: Measuring stability and change: Methodological issues in quality of life studies. Social Indicators Research, 130(3), 1297.
Maglio, S. J., & Trope, Y. (2011). Scale and construal: How larger measurement units shrink length estimates and expand mental horizons. Psychonomic Bulletin & Review, 18(1), 165-70.
May, C., Campbell, S., & Doyle, H. (1998). Research misconduct: A pilot study of British addiction researchers. Addiction Research & Theory, 6(4), 371-373.
Meehl, P.A. (1990). Appraising and amending theories: The strategy of Lakatosian defense and two principles that warrant using it. Psychological Inquiry, 1(2), 108-141.
Meehl, P. E. (1959). Some ruminations on the validation of clinical procedures. Canadian Journal of Psychology/Revue Canadienne de Psychologie, 13(2), 102.
Meehl, P. E. (1959). Structured and projective tests: Some common problems in validation. Journal of Projective Techniques, 23(3), 268-272.
Meehl, P.A. (1967). Theory-testing in psychology and physics: A methodological paradox. Philosophy of Science, 34, 103-115.
Meehl, P. E. (1990). Why summaries of research on psychological theories are often uninterpretable. Psychological Reports, 66(1), 195-244.
Meehl, P. E., & Waller, N. G. (2002). The Path Analysis Controversy: A new statistical approach to strong appraisal of verisimilitude. Psychological Methods, 7(3), 283.
Meyers, L. S., Gamst, G., & Guarino, A. J. (2013). Applied multivariate research: Design and interpretation (2nd ed. ed.) Sage Publications, Inc, Thousand Oaks, CA.
Michell, J. (1997). Quantitative science and the definition of measurement in psychology. British Journal of Psychology, 88(3), 355-383.
Miller, A. G., McHoskey, J. W., Bane, C. M., & Dowd, T. G. (1993). The attitude polarization phenomenon: Role of response measure, attitude extremity, and behavioral consequences of reported attitude change. Journal of Personality and Social Psychology, 64(4), 561.
Mills, E. J., Kanters, S., Thorlund, K., Chaimani, A., Veroniki, A. A., & Ioannidis, J. P. (2013). The effects of excluding treatments from network meta-analyses: survey. BMJ, 347.
Morison, T., Gibson, A. F., Wigginton, B., & Crabb, S. (2015). Online research methods in psychology: Methodological opportunities for critical qualitative research. Qualitative Research in Psychology, 12(3), 223-232.
Mumford, J. A., Davis, T., & Poldrack, R. A. (2014). The impact of study design on pattern estimation for single-trial multivariate pattern analysis. NeuroImage,103, 130-138.
Murad, M. H., Montori, V. M., Ioannidis, J. P., Jaeschke, R., Devereaux, P. J., Prasad, K., … & Guyatt, G. (2014). How to read a systematic review and meta-analysis and apply the results to patient care: users’ guides to the medical literature. JAMA, 312(2), 171-179.
Palakshappa, N., & Gordon, M. E. (2006). Using a multi-method qualitative approach to examine collaborative relationships. Qualitative Market Research, 9(4), 389-403.
Parker, I. (2013). Discourse analysis: Dimensions of critique in psychology. Qualitative Research in Psychology, 10(3), 223-239.
Patel, C. J., & Ioannidis, J. P. (2014). Studying the elusive environment in large scale. JAMA, 311(21), 2173-2174.
Patsopoulos, N. A., & Ioannidis, J. P. (2009). The use of older studies in meta-analyses of medical interventions: a survey. Open Medicine, 3(2), e62.
Pennycook, G., Cheyne, J. A., Barr, N., Koehler, D. J., & Fugelsang, J. A. (2015). On the reception and detection of pseudo-profound bullshit. Judgement and Decision Making, 10(6), 549-563.
Plonsky, L., & Gass, S. (2011). Quantitative research methods, study quality, and outcomes: The case of interaction research. Language Learning, 61(2), 325-366.
Plunk, V. (2013). Who’s afraid of peer review?. Science, 342, 60-65.
Prasad, V., & Ioannidis, J. P. (2014). Evidence-based de-implementation for contradicted, unproven, and aspiring healthcare practices. Implement Sci, 9(1), 5908-9.
Robinson, K. A., & Goodman, S. N. (2011). A systematic examination of the citation of prior research in reports of randomized, controlled trials. Annals of Internal Medicine, 154(1), 50-55.
Schmittmann, V. D., Cramer, A. O. J., Waldorp, L. J., Epskamp, S., Kievit, R. A., & Borsboom, D. (2013). Deconstructing the construct: A network perspective on psychological phenomena. New Ideas in Psychology, 31(1), 43-53.
Schooler, J. (2011). Unpublished results hide the decline effect. Nature, 470(7335), 437.
Schuemie, M. J., Ryan, P. B., Suchard, M. A., Shahn, Z., & Madigan, D. (2014). Discussion: An estimate of the science-wise false discovery rate and application to the top medical literature. Biostatistics, 15(1), 36-39.
Simmons, J.P., Nelson, L.D., & Simonsohn, U. (2011). False-positive psychology: Undisclosed flexibility in data collection and analysis allows presenting anything as significant. Psychological Science, 22, 1359-1366.
Slater, M. D., Snyder, L., & Hayes, A. F. (2006). Thinking and modeling at multiple levels: The potential contribution of multilevel modeling to communication theory and research. Human Communication Research, 32(4), 375-384.
Stack, C. B., Localio, A. R., Griswold, M. E., Goodman, S. N., & Mulrow, C. D. (2013). Handling of rescue and missing data affects synthesis and interpretation of evidence: the sodium–glucose cotransporter 2 inhibitor example. Annals of Internal Medicine, 159(4), 285-288.
Sun, X., Ioannidis, J. P., Agoritsas, T., Alba, A. C., & Guyatt, G. (2014). How to use a subgroup analysis: users’ guide to the medical literature. JAMA, 311(4), 405-411.
Tavare, A. (2011). Managing research misconduct: is anyone getting it right?. BMJ, 343.
ter Riet, G., Chesley, P., Gross, A. G., Siebeling, L., Muggensturm, P., Heller, N., … & Puhan, M. A. (2013). All That Glitters Isn’t Gold: A Survey on Acknowledgment of Limitations in Biomedical Studies. PloS One, 8(11), e73623.
Tian, L., Alizadeh, A. A., Gentles, A. J., & Tibshirani, R. (2014). A Simple Method for Estimating Interactions Between a Treatment and a Large Number of Covariates. Journal of the American Statistical Association, 00-00.
Trikalinos, N. A., Evangelou, E., & Ioannidis, J. P. (2008). Falsified papers in high-impact journals were slow to retract and indistinguishable from nonfraudulent papers. Journal of Clinical Epidemiology, 61(5), 464-470.
Trinquart, L., Ioannidis, J. P., Chatellier, G., & Ravaud, P. (2014). A test for reporting bias in trial networks: simulation and case studies. BMC Medical Research Methodology, 14(1), 112.
Trope, Y., & Liberman, N. (2010). Construal-level theory of psychological distance. Psychological Review, 117(2), 440-463.
Tzoulaki, I., Ebbels, T. M., Valdes, A., Elliott, P., & Ioannidis, J. P. (2014). Design and analysis of metabolomics studies in epidemiologic research: a primer on-omic technologies. American Journal of Epidemiology, 180(2), 129-139.
Tzoulaki, I., Seretis, A., Ntzani, E. E., & Ioannidis, J. P. (2014). Mapping the expanded often inappropriate use of the Framingham Risk Score in the medical literature. Journal of Clinical Epidemiology, 67(5), 571-577.
Wagenmakers, E.J., Wetzels, R., Borsboom, D., van der Maas, H.L.J., & Kievit, R.A. (2012). An agenda for purely confirmatory research. Perspectives on Psychological Science, 7, 632-638.
Waller, N. G., & Meehl, P. E. (2002). Risky tests, verisimilitude, and path analysis. Psychological Methods, Vol 7(3), Sep 2002, 323-337.
Wernerova, M., & Hudlicky, T. (2010). On the Practical Limits of Determining Isolated Product Yields and Ratios of Stereoisomers: Reflections, Analysis, and Redemption. Synlett, 2010(18), 2701-2707.
Zentall, T. R. (2008). Critical thinking in psychology. The Psychological Record, 58(1), 156-158.
Zimmerman, D. E. (2004). Qualitative communication research Methods/Longitudinal Research/Confidence Intervals/Reading statistics and research. Technical Communication, 51(3), 437-439.
Zintzaras, E., & Ioannidis, J. P. (2014). HELOW: A program for testing extreme homogeneity in meta-analysis. Computer Methods and Programs in Biomedicine, 117(2), 383-386.