Science Correcting Itself | Alternative Explanations, Interpretations, and Conclusions

Alternative Explanations, Interpretations, and Conclusions

Correction is an important element of scientific progress.  In the history of scientific investigations, it has been common for widely-accepted ideas to be overturned.  Literature contains many instances of such self-correction, though the corrections are sometimes not as widely known as the original findings.  Here, we catalog some corrections, all of which are instances science can be proud of.  Investigators are invited to submit to us more such examples to be listed here and to illustrate more instances in which science was successfully self-correcting.

Voter Fraud: Non-citizens Voting in American Elections

Richman et al. presents data suggesting that some non-citizens vote in U.S. elections despite being ineligible. The study further argues that non-citizens favor Democratic candidates over Republican candidates and that non-citizen voting was likely significant enough to change the 2008 election outcomes of both Electoral College votes and Congress. Ansolabehere et. al do a statistical analysis of Richman et al. and finds that the study is flawed because sampling errors that would normally be acceptable can significantly alter the findings of a study if making inferences about low-probability events from large-N survey data. Ansolabehere et al. concludes that the Richman et al. study is highly flawed and that the non-citizen voting rate in the U.S is likely 0. Richman responded to both Trump’s and Ansolabehere et al. mostly sticking by his original findings. He states that Trump has been exaggerating his research to make claims that are unsupported by the data. Even if all of the data in the study is accepted as correct Clinton would have received 835,000 fraudulent votes. Even without these votes she would have still won the popular vote by over 2 million votes. Richman also, however, argues that those on the left are equally unfounded in stating that the study has no value.

Original Article

Richman, J.T., Chattha, G.A., & Earnest, D.C. (2014). Do non-citizens vote in U.S. elections? Elsevier: Electoral Studies Journal, 36, 149-157. 

Alternative Views

Ansolabehere, S., Luks, S., & Schaffner, B.F. (2015). The perils of cherry picking low frequency events in large sample surveys. Elsevier: Electoral Studies Journal, 40, 409-410. 

Issie Lapowsky (2017). Author of Trump’s Favorite Voter Fraud Study Says Everyone’s Wrong. Wired 

The Actor-Observer Difference

Jones et al and Jones and Nisbett support the Actor-Observer Hypothesis. They argue that individuals will use situation or context to explain their own actions but are more likely to attribute the behavior of others to their personalities or character traits. Malle disputes these findings, concluding from a meta-analysis that the effects are largely insignificant, except in very specific circumstances.

Original Article(s)

Jones, E.E., Rock, L., Shaver, K.G., Goethals, G.R., & Ward, L.M. (1968). Pattern of performance and ability attribution: An unexpected primacy effect. Journal of Personality and Social Psychology, 10(4), 317-340. 

Jones, E.E., & Nisbett, R.E. (1972). The actor and the observer: Divergent perceptions of the causes of behavior. In E.E. Jones, D.E. Kanouse, H.H. Kelley, R.E. Nisbett, S. Valins, & B. Weiner (Eds.) Attribution: Perceiving the Causes of Behavior (pp. 79-94). New York, NY: General Learning Press.

Alternative View

Malle, B.F. (2006). The actor–observer asymmetry in attribution: A (surprising) meta-analysis. Psychological Bulletin, 132(6), 895-919. 

Attitude Polarization

Lord et al claim that people with strong or extreme opinions about a subject will interpret evidence related to that subject through a biased lens, seeking only to confirm the position they already hold. Miller et al complicate that theory, showing the unreliability of self-reported attitude changes. Kuhn and Lao also disagree with Lord et al, arguing that true attitude polarization is relatively rare, and is more likely to reflect undecided individuals moving towards choosing a position on an issue rather than people who have decided opinions becoming more extreme in their views.

Original Article

Lord, C.G., Ross, L., & Lepper, M.R. (1979). Biased assimilation and attitude polarization: The effects of prior theories on subsequently considered evidence. Journal of Personality and Social Psychology, 37, 11, 2098-2109. 

Alternative View(s)

Miller, A.G., McHoskey, J.W., Bane, C.M., & Dowd, T.G. (1993). The attitude polarization phenomenon: Role of response measure, attitude extremity, and behavioral consequences of reported attitude change. Journal of Personality and Social Psychology, 64, 4, 561-574. 

Kuhn, D., & Lao, J. (1996). Effects of evidence on attitudes: Is polarization the norm? Psychological Science, 115-120. 

Automaticity and Social Behavior

Bargh et al argue that priming can subconsciously affect subjects’ subsequent social behavior. They show that subjects exposed to rude prime were more likely to be rude towards the experimenter, subjects who received the elderly prime walked more slowly, and subjects who were primed with subliminal images of African-Americans were more likely to behave aggressively. Lakens conducted a p-curve analysis of social priming literature, finding that while the evidence for one experiment stands, the evidence for elderly priming has been p-hacked and may not represent a significant effect.

Original Article

Bargh, J.A., Chen, M., & Burrows, L. (1996). Automaticity of social behavior: Direct effects of trait construct and stereotype activation on action. Journal of Personality and Social Psychology, 71, 230-244.

Alternative View

Lakens, D. (2014). Professors are not elderly: Evaluating the evidential value of two social priming effects through p-curve analyses. Available at SSRN 2381936. ​

Climate Skepticism and Belief in Bizarre Conspiracy Theories

Lewandowsky et al present the argument that support for free market policies and belief in conspiracy theories are both indicators that an individual is a climate change doubter. Dixon and Jones conduct analyses of the data in Lewandowsky et al and find that the evidence actually suggests climate change deniers are not necessarily likely to believe in conspiracy theories, and that they are less likely to believe in conspiracy theories than people who are unsure about climate change. Jussim et al also dispute the conclusions made by Lewandowsky et al, arguing that there is no evidence to support the assertion that climate change doubters also believe that the moon landing was faked, which is the claim made by the title of the paper.

Original Article

Lewandowsky, S., Oberauer, K., & Gignac, G. E. (2013). NASA faked the moon landing – therefore, (climate) science is a hoax an anatomy of the motivated rejection of science. Psychological Science, 24, 622-633. 

Correction(s)

Dixon, R. M., & Jones, J. A. (2015). Conspiracy ideation as a predictor of climate-science rejection: An alternative analysis. Psychological Science, 26, 664-666. 

Jussim, L., Crawford, J.T., Stevens, S.T., Anglin, S.M., & Duarte, J.L. (in press). Can high moral purposes undermine scientific integrity? To appear in J. Forgas, P. van Lange, & L. Jussim (eds), The Sydney Symposium on the Social Psychology of Morality. 

Implicit Association Test as a Predictor of Discrimination

McConnell and Leibold, Greenwald et al, and Heider and Skowronski provide evidence supporting the validity and accuracy of the Implicit Association Test (IAT)’s ability to detect prejudices and biases that subjects do not express consciously. Blanton et al reanalyze the data from McConnell and Leibold, concluding that the IAT is not as effective as McConnell and Leibold claim. Blanton and Mitchell reanalyze the data from Heider and Skowronski, and find that the evidence does not actually support the original conclusions. Oswald et al conduct a meta-analysis of studies on the IAT and report that the test does not predict most behavior or beliefs accurately.

Original Article(s)

McConnell, A. R., & Leibold, J. M. (2001). Relations among the Implicit Association Test, discriminatory behavior, and explicit measures of racial attitudes. Journal of Experimental Social Psychology, 37, 435– 442 

Greenwald, A. G., Poehlman, T. A., Uhlmann, E. L., & Banaji, M. R. (2009). Understanding and using the Implicit Association Test: III. Meta-analysis of predictive validity. Journal of Personality and Social Psychology, 97, 17– 41. doi:10.1037/a0015575 

Heider, J., & Skowronski, J. (2007). Improving the predictive validity of the Implicit Association Test. North American Journal of Psychology, 9, 53-76. 

Correction(s)

Blanton, H., Jaccard, J., Klick, J., Mellers, B., Mitchell, G., & Tetlock, P. E. (2009). Strong claims and weak evidence: Reassessing the predictive validity of the IAT. Journal of Applied Psychology, 94, 567–582. 

Blanton, H., & Mitchell, G. (2011). Reassessing the Predictive Validity of the IAT II: Reanalysis of Heider & Skowronski (2007). North American Journal of Psychology, 13, 1, 99-106. 

Oswald, F. L., Mitchell, G., Blanton, H., Jaccard, J., & Tetlock, P. E. (2013). Predicting ethnic and racial discrimination: A meta-analysis of IAT criterion studies. Journal of Personality and Social Psychology, 105, 171–192. 

Mere Prediction Effect

Williams et al conducted a study demonstrating that the question-behavior effect holds true for health-related issues. They claim to show that simply asking students whether or not they plan to use drugs in the future increases their likelihood of doing so. Schneider et al re-examine the data and find two major flaws in the original analysis. They argue that the t-test used in the study is misleading because of the skewed data sample, and that the small number of extreme outliers further distorts the data.

Original Article

Williams, P., Block, L.G., & Fitzsimons, G.J. (2006). Simply asking questions about health behaviors increases both healthy and unhealthy behaviors. Social Influence, 1(2), 117– 127. 

Correction

Schneider, D., Tahk, A., & Krosnick, J.A. (2007). Reconsidering the impact of behavior prediction questions on illegal drug use: The importance of using proper analytic methods. Social Influence, 2(3), 178-196.

Political Conservatism, Uncertainty, and Threat

Jost et al argue that political conservatism appeals to individuals based on personality, needs and anxieties, and ideology. Authoritarian personalities, dogmatism, death anxiety, and desire for stability, among other attributes, are often connected to political conservatism. Van Hiel et al disagree, contending that there exist only weak to moderate links between social cognition and political conservatism and criticizing Jost et al’s inclusion of studies that use self-report measures.

Original Article

Jost, J. T., Glaser, J., Kruglanski, A. W., & Sulloway, F. (2003). Political conservatism as motivated social cognition. Psychological Bulletin, 129, 339–375. 

Alternative View

Van Hiel, A., Onraet, E., & De Pauw, S. (2010). The relationship between social-cultural attitudes and behavioral measures of cognitive style: A meta-analytic integration of studies. Journal of Personality, 78, 1765-1800. 

The Power of the Situation

Ross and Nisbett and Zimbardo claim that the power of situation is strongest in determining behavior; most people will behave in predictable ways when placed into circumstances that force them to do so. Kenrick and Funder and Roberts et al emphasize the importance of individual personality in influencing behavior and attitudes, showing that that an individual’s character is a significant determinant of his or her actions. Funder and Fleeson, however, contend that both situational and dispositional factors should be integrated and that their interaction should be examined.

Original Article(s)

Ross, L., & Nisbett, R. (1991). The person and the situation: Perspectives of social psychology. New York, NY: McGraw-Hill.

Zimbardo, P. (2007). The Lucifer effect: understanding how good people turn evil. New York, NY: Random House.

Alternative Views

Kenrick, D.R., & Funder, D.C. (1988). Profiting from controversy: Lessons from the person-situation debate. American Psychologist, 43(1), 23-34. 

Roberts, B.W., Kuncel, N.R., Shiner, R., Caspi, A., & Goldberg, L.R. (2007). The power of personality: The comparative validity of personality traits, socioeconomic status, and cognitive ability for predicting important life outcomes. Perspectives on Psychological Science, 2(4), 313-345. 

Funder, D. C., & Furr, R. M. (2008). Persons, situations, and person-situation interactions. Handbook of Personality: Theory and research, 3, 568-580 

Fleeson, W. (2004). Moving personality beyond the person-situation debate the challenge and the opportunity of within-person variability. Current Directions in Psychological Science, 13(2), 83-87 

People are not Bayesian Reasoners

Khaneman and Tversky demonstrate that people make judgments according to representativeness and are most likely to make a prediction based on the representativeness of the evidence, no matter how unlikely or extreme that result is. Griffiths and Tenenbaum qualify that claim, however, arguing that people also tend to combine representativeness with probability and rational reasoning.

Original Article

Kahneman, D., & Tversky, A. (1973). On the psychology of prediction. Psychological Review, 80(4), 237-251. 

Alternative View

Griffiths, T.L., & Tenenbaum, J.B. (2006). Optimal predictions in everyday cognition. Psychological Science, 17(9), 767-773. 

Preferential Search for Hypothesis-Confirming Information

Snyder and Swann claim that when people are given information about a subject, they will ask that subject questions in a biased way in order to confirm what they think they already know. However, Trope and Bassok, Tope et al, Skov and Sherman, and Devine et al find that there is no such tendency for individuals ask biased questions or seek to confirm their hypotheses at the expense of finding out the truth.

Original Article

Snyder, M., & Swann, W. B. Jr. (1978). Hypothesis-testing processes in social interaction. Journal of Personality and Social Psychology, 36, 1202-1212.

Alternative View(s)

Trope, Y., & Bassok, M. (1982). Confirmatory and diagnosing strategies in social information gathering. Journal of Personality and Social Psychology, 43, 22-34. 

Trope, Y., Bassok, M., & Alon, E. (1984). The questions lay interviewers ask. Journal of Personality, 52, 90-106. 

Skov, R. B., & Sherman, S. J. (1986). Information gathering processes: Diagnosticity, hypothesis confirmatory strategies, and perceived hypothesis confirmation. Journal of Experimental Social Psychology, 22, 93-121. 

Devine, P. G., Hirt, E. R., & Gehrke, E. M. (1990). Diagnostic and confirmation strategies in trait hypothesis testing. Journal of Personality and Social Psychology, 58, 952-963. 

The Power of Unconscious Processes

Bargh and Chartrand argue that humans mostly process their environment and the information that they receive subconsciously, and for the large part are not aware of shaping their perceptions of the world. Newell and Shanks, however, criticize many of the conclusions made by researchers in the field and claim that the available data actually supports the theory that humans process their surroundings consciously.

Original Article

Bargh, J.A., & Chartrand, T.L. (1999). The unbearable automaticity of being. American Psychologist, 54(7), 462-479. 

Correction

Newell, B.R., & Shanks, D.R. (2014). Unconscious influences on decision making: A critical review. Behavioral and Brain Sciences, 37(1), 1-19. 

Stanford Prison Experiment

Haney and Zimbardo’s study demonstrated the power of situational forces in determining behavior by simulating a prison; the power dynamic between the guards and prisoners led the former group to oppress and mistreat the latter group, and individual personality differences could not explain the vast majority of the behavior of subjects. Fromm criticizes the manner in which the Stanford Prison Experiment was conducted and suggests that its results do not exclusively endorse the theory of the power of situation, but may also be a function of the participants’ personalities. Banuazizi and Movahedi argue that the original study did not adequately recreate the environment of a real prison, and that participants behaved in a way so as to confirm what they believed to be the hypothesis and to fulfill the preconceived roles of guard and prisoner that they held coming into the experiment. Carnahan and McFarlan claim that the original experiment should actually be interpreted in terms of person-situation interactionism, rather than looking purely at the situation.

Original Article

Haney, C., Banks, W. C., & Zimbardo, P. G. (1973). Interpersonal dynamics in a simulated prison. International Journal of Criminology and Penology, 1, 69-97.

Alternative Views

Fromm, E. (1973). The Anatomy of Human Destructiveness Holt, Rinehart, & Winston: New York, NY.

Banuazizi, A., & Movahedi, S. (1975). Interpersonal dynamics in a simulated prison: A methodological analysis. The American Psychologist, 30, 2, 152-160. 

Carnahan, Thomas; Sam McFarland (2007). Revisiting the Stanford prison experiment: could participant self-selection have led to the cruelty? Personality and Social Psychology Bulletin 33 (5), 603–14. 

Theory of Reasoned Action

Fishbein advocates for the Theory of Reasoned Action, arguing that behavioral intention can be predicted through an individual’s attitude towards that behavior, but not by character or values. However, Budd claims that depending on the design of a questionnaire asking subjects about their attitudes, subjects may adjust their answers to fit with Fishbein’s Theory of Reasoned Action.

Original Article

Fishbein, M. (1979). A theory of reasoned action: Some applications and implications. Nebraska Symposium on Motivation, 27, 65-116. 

Alternative View

Budd, R.J. (1987). Response bias and the theory of reasoned action. Social Cognition, 5(2), 95-107.