Last week, we talked about how we can evaluate whether Chris Froome doped to win the 2013 Tour de France. Let’s retake the issue where we left it.
Building the diagnostic map
With our problem-solving approach, we use a diagnostic map (or a diagnostic issue tree) to break down the stem question into its various components before summarizing those in a set of independent and collectively exhaustive (ICE) hypotheses. In this case we came up with 3 hypotheses:
- Hypothesis 1: Christopher Froome won the 2013 TdF without cheating
- Hypothesis 2: Christopher Froome won the 2013 TdF because taking some performance-enhancing drugs (i.e., doping) gave him the edge to beat his competition
- Hypothesis 3: Christopher Froome won the 2013 TdF because cheating (in a different way than doping) gave him the edge to beat his competition
(And those are indeed independent and collectively exhaustive and not mutually exclusive and collectively exhaustive (MECE) because hypotheses 2 and 3 can both happen at the same time. (For more on ICE/MECE, see here.)
There are various scenarios that might fall into hypothesis 3: maybe Froome used a bicycle that is lighter than the 6.8kg minimum limit imposed by the UCI, or maybe he had a small engine in his downtube, like Cancellara was rumored to have a couple of years back. There are other potential explanations (maybe he cut the course, or maybe he received unauthorized external aid, or maybe his bike’s geometry doesn’t comply with the regulation), but I haven’t seen any evidence that might suggest that this is the case. Therefore I chose to discard hypothesis 3 and concentrate on the first two ones. The reasoning is that by doing so, I can dedicate more firepower to testing the first two but if, after analyzing them, I find that neither one can explain the full body of evidence satisfactorily, I will need to come back to this one.
So, now I’m left with two hypotheses, and in this new diagnostic space, those are truly MECE: either Froome won without cheating or he doped.
Complex problems and deception
This is a complex problem because, if we assume that Froome cheated, then some level of deception is involved. That is, if he cheated, he will most likely pretend that he didn’t.
This is the type of problems that detectives, judges, and parents see all the time (“I swear, mom, it wasn’t me who stole Maggie’s toy!”). It’s also the types of problems that intelligence analysts deal with. The CIA’s preferred approach for this sort of problems is the Analysis of Competing Hypotheses. The approach was developed in the 1970s by one of their analysts, Richard Heuer, and it has its supporters. But it has also been criticized, perhaps most notably in a 2010 report of the National Research Council. Having used the method, I’ve also found it to have limitations.
An alternative is to use hypothesis mapping, a method quite similar to using issue trees/maps. I’ve tried to apply it to this case (see here) but I found that the resulting map is a bit too large and it’s easy to get lost. So I went with a third way.
This one starts with collecting all the pieces of evidence about the case and examining that body of evidence sequentially under each hypothesis to understand what additional assumptions are necessary to explain the full body of evidence. And this is the key part, we need to explain all the evidence.
So I’ve spent the last week or so collecting evidence related to Froome, his team, and his performance in the few years leading to this year’s Tour de France. It’s summarized in the figure below. Note: I’m not a cycling specialist, and, while I tried to be as unbiased as possible, I might not have succeeded, so if you think this needs any correction, please let me know.
In Part 3, we’ll look at how to consider this body of evidence under each of our two remaining hypotheses.