03/18/2010 05:12 am ET Updated May 25, 2011

Anecdotes Are Weak Evidence

Dr. Norbert Gleicher writes a piece in The Wall Street Journal that roundly criticizes the proposed role of expert panels that would make determinations about what treatments to cover by using comparative-effectiveness research. Like many opponents of the idea he writes "the idea of inserting a government panel between patients and physicians remains contentious..." and "evidence-based medicine has some value, but it can provide misleading information." He then goes on to discuss at length the biases inherent in the process of conducting research, getting published in the peer-reviewed literature, and even conducting the meta-analyses central to identifying evidence-based guidelines. He then uses a personal anecdote to make his case.

So many things are wrong with his piece. First of all, the expert panel would not be a "government" panel, but a panel of medical experts commissioned by the government. Those two things are quite different. Furthermore, would one prefer that a panel of non-experts be convened? Or is it that we need no panel at all? I suppose the old adage "Two heads are better than one" no longer applies. Of course, we already have our allowable treatments meted out to us by the insurance companies. I guess it does make sense to have corporate profiteering trump medical expertise when trying decide which medical interventions work the best.

And what about evidence-based guidelines? You know, the more I think about it, I guess I really would prefer to go to my doctor and have her say "Well, Brad, we really have no reason to believe that this treatment will work. I mean, there's not any evidence for it, but we're just going to give it a try, okay?" Besides, the decisions of the panel aren't legally binding. My doctor can provide me with any treatment deemed necessary. That's why they're called guidelines and not requirements.

My real beef, though, is with Dr. Gleicher's use of a single personal anecdote to refute the worth of volumes of medical research. Can research studies be biased? Absolutely. Can anecdotes be completely misguiding? You bet. Exceptions to the rule can always be found, but that doesn't mean that we should be bending to accommodate the "exceptions" in every instance at the expense of identifying the "rule" from which many more persons are likely to benefit. Is that a utilitarian perspective? It sure is, and I think it's the best way to design a system that's intended to apply to everyone as a whole.

Read or Subscribe to Wright on Health to see what else I have to say or contact me if you're interested in writing for the blog.