The quality of your analysis rests on two pillars—good thinking and good evidence—and evaluating the quality of evidence can be hard to do.

In fact, taken individually or as a body, evidence is likely to be incomplete, inconclusive, ambiguous, dissonant, and not perfectly credible (for a discussion, see Strategic Thinking, pp. 97–98).

Let's focus on that last point: Not all evidence is equally trustworthy. You should give more weight to higher-quality evidence. And, yes, expert opinion is the least trustworthy type. Here's a shorthand to help you stay afloat—a downloadable graphic that you can print out and keep handy.

You should trust most evidence from level 1, which is gathered from at least one properly designed randomized controlled trial. In plain English, this mouthful indicates that for the experiment /trial, participants are assigned randomly to the group receiving the treatment or to the control group, which may receive a placebo. 

But it is difficult to control all aspects of the environments in which we make organizational decisions, and you will probably be required to consider evidence that doesn't meet this high-quality standard. For instance, you might not be able to randomize the trial (level 2).

As you relax further constraints, you drop to levels 3 and 4 until you hit level 5, which includes reports from experts. That's right, experts' opinions are at the bottom of the trustworthiness pyramid.

Using evidence to make better decisions is a central tenet of our approach to problem solving, and if you're interested in the subject, you'll find more ideas throughout this site, including piloting new initiatives to reduce the impact of unforeseen factors during implementation, improving your innovative thinking by leveraging analogies, and looking for excellence in lieu of perfection.

References

Barends, E. G. R. (2015). "In search of evidence: empirical findings and professional perspectives on evidence-based management."

Barends, E. and D. M. Rousseau (2018). Evidence-based management: How to use evidence to make better organizational decisions, Kogan Page Publishers.

Chevallier, A. (2016). Strategic Thinking in Complex Problem Solving. Oxford, UK, Oxford University Press, pp. 88–89, 97–98.

Gauch, H. G. (2003). Scientific method in practice, Cambridge University Press.

Rousseau, D. M. and E. G. Barends (2011). "Becoming an evidence‐based HR practitioner." Human Resource Management Journal
21(3): 221-235.

Pfeffer, J. and R. I. Sutton (2006). "Evidence-based management." Harvard business review
84(1): 62.

Tecuci, G., et al. (2011). "Toward a Computational Theory of Evidence-based Reasoning." 18th International Conference on Control Systems and Computer Science, University Politehnica of Bucharest.

Tecuci, G., et al. (2014). "Computational approach and cognitive assistant for evidence–based reasoning in intelligence analysis." International Journal of Intelligent Defence Support Systems
5(2): 146-172.