Statistical Forensics



Statistical forensics refer to statistical techniques that can assess the credibility or likely replicability of scientific studies.

Statistical Forensics at Work

Research 2000: Problems in plain sight

Statistical Forensics Launches a Polling Donnybrook

Benos, D. (2006). Ethics: detecting misconduct. Nature.

Bes-Rastrollo, M., Schulze, M. B., Ruiz-Canela, M., & Martinez-Gonzalez, M. A. (2013). Financial conflicts of interest and reporting bias regarding the association between sugar-sweetened beverages and weight gain: a systematic review of systematic reviews. PLoS Med, 10(12), e1001578.

Bolton, R. J., & Hand, D. J. (2002). Statistical fraud detection: A review. Statistical science, 17(3), 235-255.

Bierman, D. J., Spottiswoode, J. P., & Bijl, A. (2016).Testing for questionable research practices in a meta-analysis: An example from experimental parapsychology. PloS one, 11(5), e0153049.

Broome, M. E., Pryor, E., Habermann, B., Pulley, L., & Kincaid, H. (2005). The Scientific Misconduct Questionnaire—Revised (SMQ-R): Validation and psychometric testing. Accountability in Research, 12(4), 263-280.

Chambers, C. D. (2013). Registered Reports: A new publishing initiative at Cortex [Editorial]. Cortex, 49(3), 609-610.

Chan, A. W., Hróbjartsson, A., Haahr, M. T., Gøtzsche, P. C., & Altman, D. G. (2004). Empirical evidence for selective reporting of outcomes in randomized trials: comparison of protocols to published articles. Jama, 291(20), 2457-2465.

Dwan, K., Altman, D. G., Clarke, M., Gamble, C., Higgins, J. P., Sterne, J. A., … & Kirkham, J. J. (2014). Evidence for the selective reporting of analyses and discrepancies in clinical trials: a systematic review of cohort studies of clinical trials. PLoS Med, 11(6), e1001666.

Fang, F. C., Steen, R. G., & Casadevall, A. (2012). Misconduct accounts for the majority of retracted scientific publications. Proceedings of the National Academy of Sciences, 109(42), 17028-17033.

Giardino, J. (2014). Caught by the numbers. Fraud Magazine.

Gross, C. (2016). Scientific misconduct. Annual review of psychology, 67, 693-711.

Gupta, A. (2013). Fraud and misconduct in clinical research: a concern. Perspectives in Clinical Research, 4(2), 144-147.

Hannink, G., Gooszen, H. G., & Rovers, M. M. (2013). Comparison of registered and published primary outcomes in randomized clinical trials of surgical interventions. Annals of surgery, 257(5), 818-823.

Inzlicht, M. (2016). Check yourself again. Toronto Laboratory For Social Neuroscience.

John, L. K., Loewenstein, G., & Prelec, D. (2012). Measuring the prevalence of questionable research practices with incentives for truth telling. Psychological science, 23(5), 524-532.

Kirkham, J. J., Dwan, K. M., Altman, D. G., Gamble, C., Dodd, S., Smyth, R., & Williamson, P. R. (2010). The impact of outcome reporting bias in randomised controlled trials on a cohort of systematic reviews. Bmj, 340, c365.

Klimek, P., Yegorov, Y., Hanel, R., & Thurner, S. (2012). Statistical detection of systematic election irregularities. Proceedings of the National Academy of Sciences of the United States of America, 109(41), 16469-16473.

Mathieu, S., Boutron, I., Moher, D., Altman, D. G., & Ravaud, P. (2009). Comparison of registered and published primary outcomes in randomized controlled trials. Jama, 302(9), 977-984.

Neale, A. V., Dailey, R. K., & Abrams, J. (2010). Analysis of citations to biomedical articles affected by scientific misconduct. Science and engineering ethics, 16(2), 251-261.

Nevitt, S. J., Marson, A. G., Davie, B., Reynolds, S., Williams, L., & Smith, C. T. (2017). Exploring changes over time and characteristics associated with data retrieval across individual participant data meta-analyses: systematic review. bmj, 357, j1390.

Pryor, E. R., Habermann, B., & Broome, M. E. (2007). Scientific misconduct from the perspective of research coordinators: a national survey. Journal of Medical Ethics, 33(6), 365-369.

Reid, E. K., Tejani, A. M., Huan, L. N., Egan, G., O’Sullivan, C., Mayhew, A. D., & Kabir, M. (2015). Managing the incidence of selective reporting bias: a survey of Cochrane review groups. Systematic reviews, 4(1), 85.

Roseman, M., Milette, K., Bero, L. A., Coyne, J. C., Lexchin, J., Turner, E. H., & Thombs, B. D. (2011). Reporting of conflicts of interest in meta-analyses of trials of pharmacological treatments.Jama, 305(10), 1008-1017.

Roseman, M., Turner, E. H., Lexchin, J., Coyne, J. C., Bero, L. A., & Thombs, B. D. (2012). Reporting of conflicts of interest from drug trials in Cochrane reviews: cross sectional study. Bmj, 345, e5155.

Rothwell, P. M. (2005). External validity of randomised controlled trials:“to whom do the results of this trial apply?”. The Lancet, 365(9453), 82-93.

Saini, P., Loke, Y. K., Gamble, C., Altman, D. G., Williamson, P. R., & Kirkham, J. J. (2014). Selective reporting bias of harm outcomes within studies: findings from a cohort of systematic reviews. bmj, 349, g6501.

Schulz, K. F., Chalmers, I., Hayes, R. J., & Altman, D. G. (1995). Empirical evidence of bias: dimensions of methodological quality associated with estimates of treatment effects in controlled trials. Jama, 273(5), 408-412.

Should you put your trust in medical research? (Chicago Tribune, June 8, 2015)

Stroebe, W., Postmes, T., & Spears, R. (2012). Scientific misconduct and the myth of self-correction in science. Perspectives on Psychological Science, 7(6), 670-688.

Thomas, J.R., Jr. (2015). Scientific misconduct: red flags. TheScientist.

Thompson, N. (2016). SIFT: a new tool for statistical detection of test fraud. Assessment Systems.

Tougher Action Needed in Fight Against Scientific Fraud (Laboratory Equipment, November 9, 2016)

Tie, R. (2011). Zeroing in on fraud.Fraud Magazine.

Woo, S. E., O’Boyle, E. H., & Spector, P. E. (2017). Best practices in developing, conducting, and evaluating inductive research.

Yong, E. (2012). A failed replication draws a scathing personal attack from a psychology professor. Discover magazine blog.

BPS invites readers to send (to relevant papers and links to add to this website.