Preventive Medicine: Maladies of evidence

Dr. David Katz

I just read that a former science editor at a major, global publication has concluded that science advocacy is boring, and that she would now prefer to “slaughter the sacred cows” of conviction. Leaving aside the somewhat brutal image, and the potential innocence and genuine sacredness of the cows in question, we may simply note that acquiring conviction born of science generally takes years, even decades. Disparaging it just takes a news cycle, innuendo, and a bit of click-bait.

Wherever medical news incites passions, a contagion of unintended consequences finds vulnerable hosts. If ever we are to overcome these maladies of our own, inadvertent devising, we must first understand them.

One method to rule them all. A hammer is an excellent tool; every carpenter needs one. But it makes a very poor saw, a lousy screw driver, and an utterly useless router. It’s a great tool, but it only does what it does.

The randomized controlled trial is just the same, as fundamental to the advance of medical understanding as a hammer (or nail gun) is to every carpentry project. But it only does what it does. What does it do?

For starters, RCTs only ever answer the questions that are posed; this is true across all methods of science. No method can produce useful answers to vapid questions; no quantity of science can redeem senselessness.

RCTs have very important weaknesses — and especially in the lifestyle domain. Rather than belabor those here, let me just ask you a question instead: would you be willing to let someone else “randomly” assign you to a dietary/exercise/sleep pattern for the next decade or two, rather than choosing for yourself? I don’t know anyone else who would be, either.

I heard it through the grapevine. Hard though it may be to remember or believe, there was a time when the only people who read medical journals had subscriptions, and received them in print. These were, of course, carted around on sleighs, because the wheel had not yet been invented… The audience for these incremental additions to the biomedical fund of knowledge included practitioners and scientists, and credentialed media outlets that would receive print copies in advance under embargo. Under such quaint and archaic circumstances, media coverage was, if not always spot on, generally competent and careful and considered.

Now, of course, medical studies — or at least snippets of them — are circulated digitally in real time. Everyone has access to them, and that is the only license required to opine. Opine people do, with or without relevant expertise. In fact, expertise is a hindrance, because experts feel obligated to read studies completely and carefully, and at least to attempt a bias-free interpretation. The highly partisan, non-expert knows no such constraints; and that much less so the true fanatic, fool, or Internet mercenary.

What’s good for the goose … gives the gander a pass. Challenges to the reliable interpretation of evidence don’t invite caution as they should; instead, they seem inevitably to encourage a rush to judgment in the opposing direction. Even as we are told how and why to distrust science, we are told how and why some new study is — all on its own — enough to change everything we thought we knew until this morning. Folks, it takes less than half a wit to see that both of these cannot be true.

Even truth… is (mis)judged by the company it keeps. Imagine that truly independent, impartial experts follow the evidence on a given topic (let’s say, plant-food predominant diets) where it leads. Their conclusion aligns with the native preferences of some groups, and opposes others. The default response of opponents these days is to denounce the experts, and their conclusion, as partisan and biased simply because it’s not the position they happen to favor. But of course, such dismissal obviously puts the sausage-laden cart before the horse.

The goals of evidence-based medicine are laudable. But evidence is just means; understanding is the ends. Until or unless we better manage the conscription of evidence into contagions of misinformation, we are all apt to be victims of plagues of misunderstanding.