Confirmation bias—seeking and interpreting evidence partially so as to support one’s beliefs—is so destructive that it can render your analysis useless. And chances are, you are a victim of it.
Raymond Nickerson, a psychology professor at Tufts University, found evidence of confirmation bias in a number of disciplines. (His 1998 paper published in the Review of General Psychology is both enlightening and sobering: citing hundreds of sources, he makes a compelling argument that confirmation bias is everywhere.)
Nickerson notes that the bias may occur in various ways:
- Restricting our attention to a single, favored hypothesis
- Treating preferentially evidence that supports our existing beliefs
- Looking primarily for positive cases
- Assigning too much credence to positive confirmatory instances
- Seeing what we are looking for
Him and others offer a few ideas to help you avoid confirmation bias:
Consider several hypotheses. As Chamberlin and Platt have noted, considering several hypotheses helps you not fall blindly in love with your ideas.
Focus on falsifying hypotheses. Hypotheses are strengthened when they survive the concerted attempts to disprove them by highly qualified people. Yet, as Bazerman and Moore noted, “people naturally tend to seek information that confirms their expectations and hypotheses, even when disconfirming or falsifying information is more useful.” [p. 29].
Question your confidence. We all tend to be too confident in our thinking (see here for more).
Bazerman, M. H. and D. A. Moore (2008). Judgment in managerial decision making, Wiley.
Chamberlin, T. C. (1965). “The method of multiple working hypotheses.” Science 148(3671): 754-759.
Nickerson, R. S. (1998). “Confirmation bias: a ubiquitous phenomenon in many guises.” Review of General Psychology 2(2): 175.
Platt, J. R. (1964). “Strong inference.” Science 146(3642): 347-353.