This is the headline would like to see, whenever we hear about the latest scientific research in the media. However, even with the best scientific journals, pinning down “the truth” in medicine can be a tricky business. I think that science is important in guiding our health care decisions this, but we must be wary about overreliance on so-called evidence-based medicine.
At the most basic level, any time there is a health claim, you have to ask whether it comes from a credible source or not. Testimonials or “studies” conducted by a product manufacturer that actually involve only a few subjects are not valid sources to draw any hard conclusions.
Okay, so can we rely on the well conducted studies in the major medical journals? Well, they are better, but they still need to be taken with a grain of salt. There are many sources of errors for the data and conclusions in major scientific studies:
- Bias: researchers themselves are biased to interpret their data in a way that supports their hypothesis; journals commonly exhibit publication bias, being more likely to publish studies that agree with previous research findings
- Difficulty with randomization: the “gold standard” of medical research is the randomized controlled trial (selecting research subjects at random to receive either the active therapy or placebo). However, achieving true randomization, and controlling adequately for confounding factors is more difficult than is commonly believed. These issues can lead to drawing wrong conclusions.
- The decline effect: sometimes, a new research headline will grab us with its dramatic statistics. As further research is conducted in the same area, however, it often turns out that the new therapy is not as effective as it seemed at first. There are many theories as to why this occurs — the simplest being that this is just “regression to the mean” as more data are gathered — but the significance is that if treatment decisions are made based on early evidence, it might lead us down the wrong path. One of the most dramatic examples of this is what are called “second-generation” antipsychotic medications. These medications (such as Zyprexa) were initially found to be incredibly more effective than older drugs at controlling schizophrenic symptoms. As further research has been conducted, it now appears that the newer drugs are no more effective than the old ones (and may even be less effective).
So what does this mean about our reliance on scientific research? The biggest take-home message is that we have to look at the big picture, rather than chasing down every rabbit hole of the latest headline. A healthy diet and exercise are irrefutably good for our health. Indeed, it may be argued that the most significant improvement of human health in the last 200 years was brought about by modern sanitation (clean drinking water and good sewer systems). Everything since then may just be splitting hairs.
Labos C. It Ain’t Necessarily So: Why Much of the Medical Literature Is Wrong. Medscape, 9 Sep 2014.