Archival studies assessing:
- The evidentiary value of and prevalence of p-hacking in a variety of social psychology literatures.
- The association of sample size with journal impact factor.
- Changes in sample size over time.
- The rate at which hypotheses are clearly stated and operationalized.
- How well suggested reforms have succeeded at increasing the credibility and validity of research findings.
- Add the following: Whether research that addresses highly politicized issues is more vulnerable to problematic practices than other types of research. These investigations make use of the p-curve and the incredibility index as statistical methods of analysis.
Questionnaire Studies: Research on Scientific Integrity
- Exploratory interviews to design questionnaires assessing scientists’ beliefs about the prevalence of integrity-impairing practices in their discipline.
- Cognitive pre-testing of questionnaires on small samples of researchers from two scientific disciplines – one natural science and one social science.
- Administration of questionnaires to larger samples of researchers from the same two scientific disciplines.
- Conventional pre-testing of questionnaires on small samples of researchers from the disciplines of chemistry and sociology to assess use of discipline-specific language and rate of attrition.
- Follow-up interviews with conventional pre-test participants to improve survey efficacy.
- Feasibility testing of questionnaires assessing scientists’ beliefs about the prevalence of suboptimal research practices in their discipline.
- Completing statistical analyses of the obtained data and writing up the results to be published in academic journals.