Every day, there is another medical study in the news. There's another newspaper or TV story telling us that X can cure depression or make you thinner or cause autism or whatever. And since it's a medical study, we usually think that it's true. Why wouldn't it be?
But what most people don't realize, let alone really think about, is that there might be other studies that show that X does none of those things -- and that some of those studies might never have been published.
Just this week, the journal Pediatrics released an article that perfectly demonstrates this problem. There have been a number of studies that have shown that a certain type of medication, selective serotonin reuptake inhibitors (SSRIs), can help stop the repetitive behaviors of autism, like hand-flapping or head-banging. If you were to do a search of the medical literature, as doctors and parents and patients often do, you'd think that using SSRIs is a good idea. But when researchers dug deeper, they found that there were just as many unpublished studies that showed that SSRIs didn't help. If they had all been published (they were all good enough to be published), that same search of the medical literature would have shown that using SSRIs isn't a good idea.
This is bad. We rely on studies to guide our decisions. What is going on?
The journals that publish articles certainly play a role. After all, it's cooler to publish a study that has a grabby headline, that promises an answer or a cure. That's much more likely to get readers than a study that says that something doesn't do anything at all. But it turns out that the researchers themselves play a bigger role.
Some researchers don't even write up their studies or try to publish them. You can't blame them, to some extent. If you set out to show that Wonderdrug cures Bad Disease, and it turns out that Wonderdrug does squat, that hardly seems interesting or important. Or, if you start studying Wonderdrug and then have to stop the study because of side effects or other problems with giving it, you might not write that up or try to publish it either -- after all, it wasn't finished. So the study goes into the trash bin, and you move on to the next idea. No big deal, right?
Well, if other studies are being published that say that Wonderdrug does cure Bad Disease, and don't mention any side effects or problems, it is a big deal -- because a whole other side of the story would be missing.
A companion study in Pediatrics looked at clinical research involving children and found that results weren't available for more than half the studies involving children -- because they weren't completed or weren't published.
As a pediatrician, this freaks me out. I'm making medical decisions for my patients based on less than half of the information out there?
There is a database, ClinicalTrials.gov, where researchers are supposed to "register" their studies before they even start. That way, there is a record of it -- and even if they stop it, or if it never gets published, there is a way to at least know it existed and find out more about it. Unfortunately, not all studies get registered. Many journals, including Pediatrics, won't publish a study unless it was registered -- if all journals would do that, maybe all researchers would register their studies.
This doesn't mean that you can't trust medical research. Medical research has brought us antibiotics and heart surgery and drugs that help many cancers; medical research saves and improves countless lives every single day. But it does mean you have to be a savvy and skeptical consumer of health information. It means that you need to ask questions, learn about the sources of the information, read widely and -- always -- talk to your doctor before you make a health decision based on a study.
It means that you always have to remember that there just might be another side to the story.
For more by Claire McCarthy, M.D., click here.
For more health news, click here.
Follow Claire McCarthy, M.D. on Twitter: www.twitter.com/@drClaire
Combing through food diaries looking for associations seems to be the method "du jour." Finding an association then requires clinical, double-blind, controlled experiments to test the hypothesis implied by the association. It seems those follow-up experiments are not being done and articles are published reporting on the associations. The media publishes a summary of the study and the unschooled public reads the articles, then believes that association means "cause and effect." The obesity and diabetes epidemics in this country are two sad examples of how an "association" was taken as "cause and effect," (the infamous lipid hypothesis) and everyone began to replace dietary fat with carbohydrates.