Recognize the different levels of trustworthiness for evidence

You should trust most evidence from level 1, which is gathered from at least one properly designed randomized controlled trial. In plain English, this mouthful indicates that for the experiment /trial, participants are assigned randomly to the group receiving the treatment or to the control group, which may receive a placebo. 

But it is difficult to control all aspects of the environments in which we make organizational decisions, and you will probably be required to consider evidence that doesn’t meet this high-quality standard. For instance, you might not be able to randomize the trial (level 2).

As you relax further constraints, you drop to levels 3 and 4 until you hit level 5, which includes reports from experts. That’s right, experts’ opinions are at the bottom of the trustworthiness pyramid.

Using evidence to make better decisions is a central tenet of our approach to problem solving, and if you’re interested in the subject, you’ll find more ideas throughout this site, including piloting new initiatives to reduce the impact of unforeseen factors during implementation, improving your innovative thinking by leveraging analogies, and looking for excellence in lieu of perfection.

References

Barends, E. G. R. (2015). “In search of evidence: empirical findings and professional perspectives on evidence-based management.”

Barends, E. and D. M. Rousseau (2018). Evidence-based management: How to use evidence to make better organizational decisions, Kogan Page Publishers.

Chevallier, A. (2016). Strategic Thinking in Complex Problem Solving. Oxford, UK, Oxford University Press, pp. 88–89, 97–98.

Gauch, H. G. (2003). Scientific method in practice, Cambridge University Press.

Rousseau, D. M. and E. G. Barends (2011). “Becoming an evidence‐based HR practitioner.” Human Resource Management Journal 21(3): 221-235.

Pfeffer, J. and R. I. Sutton (2006). “Evidence-based management.” Harvard business review 84(1): 62.

Tecuci, G., et al. (2011). “Toward a Computational Theory of Evidence-based Reasoning.” 18th International Conference on Control Systems and Computer Science, University Politehnica of Bucharest.

Tecuci, G., et al. (2014). “Computational approach and cognitive assistant for evidence–based reasoning in intelligence analysis.” International Journal of Intelligent Defence Support Systems 5(2): 146-172.