Having identified a set of hypotheses you need to test them. To do so, you need to conduct tests that can help you rule out some of the hypotheses. Sounds obvious, right?
Conduct the right analysis: easier said than done
Unfortunately, conducting the right analysis is not always (or perhaps even hardly ever) our preferred way of proceeding. In a classic article published in 1964 in Science, John Platt made the case that Pasteur was able to have a prolific career in a number of unrelated fields because he was particularly good at asking the right questions and conducting the right analysis to test his hypotheses. Pasteur shined because his competitors weren’t as skilled.
This inability to focus on the analysis that’s necessary isn’t limited to science. I have seen it in managerial settings both in the corporate and in the non-profit worlds: facing a complex issue, we easily resort to gathering information related to it without first asking whether it is the one actually needed to discard hypotheses (especially now that information is more accessible than ever). Next, we spend significant time and resources on interpreting it before realizing that it doesn’t really help us rule out hypotheses.
To recalibrate, start by identifying which information is needed
Instead, we need to reverse that process. Once you have developed your set of hypotheses, the first step should be to define the specific information needed to test it. This is similar to our approach for formulating the introductory flow of problems—where we weed out all the superfluous material from our situation, complication, and key question. Then gather that information; if it’s not available, decide whether to use a proxy but always refer to your original test to ensure that you have moved away from relevancy.
Platt, J. R. (1964). “Strong inference.” Science 146(3642): 347-353.