Scientific journals' tendency to favor studies with positive test results may have skewed the understanding of how effective antidepressants actually are in treating repetitive behavior, a common symptom of autism.
A review of recent trials, including several that were unpublished, suggests that publication bias may cause clinicians to believe serotonin reuptake inhibitors (SRIs) as more effective than they actually are.
"This matters because prescribers and health care professionals are reading these studies and taking them to be the gospel truth," said Dr. Erick Turner, a professor of psychiatry and pharmacology at Oregon Health and Science University in Portland. "So what appears to be the truth, may not be the truth at all."
In their analysis, published Monday in the journal Pediatrics, researchers searched scientific databases for recent randomized, double-blind, controlled trials testing the efficacy of SRIs in treating repetitive behaviors in autism.
Repetitive behaviors such as rocking, hand-flapping and saying the same words over and over are common symptoms of autism, as is the tendency to engage only in a restricted range of activities and develop deep preoccupations. Because the repetitive behaviors associated with autism overlap, in many ways, with the common symptoms of obsessive-compulsive disorder, experts have questioned whether patients might also respond to medications regularly prescribed for OCD, including certain antidepressants.
After their database search, the authors of the new study identified 10 trials tackling that question -- five published, five unpublished. A review of the published studies revealed a small, but significant, effect the drugs had in treating repetitive behaviors among autism patients.
But when the researchers adjusted the results to include the unpublished results, the effect was no longer significant.
"When we put it all together, trying to adjust for the missing trials, we realized that it is very likely that these medications are not as effective as previously thought," said Melisa Carrasco, a researcher with the neuroscience graduate program at the University of Michigan and an author on the study. "We go through a series of statistical tests that allow us to see there's a red flag."
Knowledge of the existence of publication bias, in general, is nothing new.
The Cochrane Collaboration, an international group that reviews medical evidence, states that "many researchers have shown that those studies with significant, positive results are easier to find than those with non-significant or 'negative' results."
But Carrasco believes this is among the first studies to look at drugs used to treat autism symptoms.
She and her fellow authors reached out to the researchers of the unpublished studies, she said, many of whom never responded, while several indicated they were not "interested in sharing" their results at this point.
Turner, who has written on publication bias in various areas of psychology, hypothesized that journals might turn down studies with negative results that are not going to change practice. At the same time, researchers might self-censor -- not submitting pieces they think are going to get rejected given the pressure to publish that dominates academic institutions.
"It can be stigmatizing," Turner said. "People think, 'I'll be the laughing stock of my peers if I get a negative result.'"
He added that in recent years, more and more journals have made a point to publish negative results, although he called the process slow.
The new analysis is not without significant limitations, which Carrasco was quick to point out to The Huffington Post.
Even after review, she said, researchers did not have the full scope of the data collected -- particularly from the unpublished trials -- which makes determining true efficacy difficult. Carrasco said the next steps are to continue reaching out to researchers to obtain and analyze, their full results. Another limitation to the current review is the lack of clear, specific assessments that can be used to quantify repetitive behaviors in clinical trials, which makes true comparison of results challenging.
Indeed, the experts caution that further research is necessary to truly understand how publication bias might influence our understanding of the efficacy of antidepressants in certain symptoms of autism spectrum disorders, which currently affect 1 in 88 children in the United States.
Even less clear is which options, if any, this leaves for parents determining how to best treat their children's symptoms.
"What we are saying here is that if we use standard, meta-analytical procedures to take into account publication bias, these treatments might still be effective," Carrasco said. "But they're perhaps not as effective as previously thought."