iOS app Android app More

Featuring fresh takes and real-time analysis from HuffPost's signature lineup of contributors
David Kirby

GET UPDATES FROM David Kirby
 

CDC: Vaccine Study Design "Uninformative and Potentially Misleading"

Posted: 06/20/08 07:43 PM ET

CDC Director Dr. Julie Gerberding has delivered a potentially explosive report to the powerful House Appropriations Committee, in which she admits to a startling string of errors in the design and methods used in the CDC's landmark 2003 study that found no link between mercury in vaccines and autism, ADHD, speech delay or tics.

Gerberding was responding to a 2006 report from the National Institute of Environmental Health Sciences (NIEHS), which concluded that methodology such as that used in the CDC's flagship thimerosal safety study is riddled with "several areas of weaknesses" that combine to "reduce the usefulness" of using the data in such a way.

"CDC concurs," Dr. Gerberding wrote in an undated report to Congress, (provided to me through a Capital Hill staffer) adding that her agency "does not plan to use" the database in question, the Vaccine Safety Datalink, (VSD) for any future "ecological studies" of autism (though that is exactly what they did in the 2003 study).

In fact, Gerberding's report said, use of the VSD for an ecological study now, comparing autism rates before and after the removal of most mercury from vaccines, "would be uninformative and potentially misleading."

Ecological vaccine studies are large, epidemiological analyses of risks and trends using computerized data from large populations -- in this case children enrolled at several big HMOs -- without linking results to actual individual outcomes.

CDC officials conducted at least five separate analyses of the data over a four-year period from 1999-2003. The first analysis showed that children exposed to the most thimerosal by one month of age had extremely high relative risks for a number of outcomes, compared with children who got little or no mercury: The relative risk for ADHD was 8.29 times higher; for autism, it was 7.62 times higher; ADD, 6.38 times higher; tics, 5.65 times; and speech and language delays were 2.09 more likely among kids who got the most mercury.

Over time, however, all of these risks declined into statistical insignificance, statistical inconsistency or else outright oblivion: The relative risk for autism plummeted from 7.62 in the first analysis, to 2.48 in the second version, to 1.69 in the third round, to 1.52 in the fourth, and down to nothing at all in the fifth, final, and published analysis printed in the Journal Pediatrics in November of 2003.

Vaccine officials attributed the steady drop to the elimination of "statistical noise" from the data through due diligence and the endeavor for excellence in governmental statistical analysis.

Indeed, the VSD study was the main pillar of a hugely influential 2004 report by the Institute of Medicine, which also concluded that there was no evidence of link between mercury, vaccines and autism.

To this day, public health officials routinely point to five "large epidemiological studies" representing the "highest quality science," none of which found any link to thimerosal.

In fact, the American VSD study has long been held up as the best and brightest of them all (the others were in Sweden, the UK, and two in Denmark). And this reputation has stuck in the minds of medicine and the media.

Curiously though, even the study's lead author -- Dr. Thomas Verstraeten, an employee of vaccine maker GlaxoSmithKline -- protested that the VSD study "found no evidence against an association, as a negative study would. In fact, he said that additional study was needed, which "is the conclusion to which a neutral study must come."

That's when Congress stepped in.

In 2005, a group of Senators and Representatives headed by Sen. Joe Lieberman wrote to the NIEHS (an agency of the National Institutes of Health) saying that many parents no longer trusted the CDC to conduct independent minded studies of its own vaccine program. Lieberman et al asked NIEHS to review the CDC's work on the vaccine database and report back with critiques and suggestions.

Among the official tasks given to the NIEHS panel were to "Identify the strengths and weaknesses of the VSD for evaluating the possible association between exposures to thimerosal-containing vaccines and AD/ASD." Another was to "Advise NIEHS and CDC on the feasibility of a new VSD study to compare autism rates before and after removal of thimerosal from most US childhood vaccines, using an ecologic study (an epidemiologic design where there is no linkage between individual-level data on exposure and health outcome).

The final NIEHS report was a serious and thoughtful critique of if an how the CDC improve on the design, conduct and analysis of VSD data in the future. Still, the NIEHS panel "identified several serious problems," with using VSD data for ecological studies -- including the 2003 study - something to which the agency had not responded -- until now.

In her letter to the House Appropriations Committee, the CDC Director responded directly to many -- though not all -- of the most important critiques and recommendations contained in the NIEHS panel report.

For example, the NIEHS had said the VSD data failed to account for other mercury exposures, including maternal sources from flu shots and immune globulin, as well as mercury in food and the environment.

"CDC acknowledges this concern and recognizes this limitation," the Gerberding reply says. This limitation would have applied to the 2003 Verstraeten study, as well as any future ecological analyses.

The NIEHS also questioned why CDC investigators eliminated 25% of the study population for a variety of reasons, even though this represented, "a susceptible population whose removal from the analysis might unintentionally reduce the ability to detect an effect of thimerosal." This strict entry criteria would likely lead to an "under-ascertainment" of autism cases, the NIEHS reported. Again, this would have been an issue in the Verstraeten data as well.

"CDC concurs," Gerberding wrote, again noting that its study design was "not appropriate for studying this vaccine safety topic. The data are intended for administrative purposes and may not be predictive of the outcomes studied."

Another serious problem is that the HMOs have changed the way they tracked and recorded autism diagnoses over time, including during the period when vaccine mercury levels were in decline. Such changes could "affect the observed rate of autism and could confound or distort trends in autism rates," the NIEHS warned.

"CDC concurs," Dr. Gerberding wrote again, "that conducting an ecologic analysis using VSD administrative data to address potential associations between thimerosal exposure and risk of ASD is not useful."

Read that sentence one more time. The head of the CDC is saying that its most powerful and convincing piece of exonerating evidence for thimerosal was based on anecological analysis of data whose methodology would be, in effect, "useless" for future study.

I hope everyone will read these documents, including the recommendations to make the VSD better, and the CDC's agreement with all of the suggestions. Hopefully, this data can still be used in some effective way.

As questionable as the US thimerosal methodology was, "it was an improvement on other studies, including the two in Denmark, both of which had serious weaknesses in their designs," Dr. Irva Hertz-Picciotto, Professor of Public Health at UC Davis Medical School and Chair of the NIEHS panel, told reporter Dan Olmsted at UPI.

That leaves very little for the CDC to go on in terms of proving that thimerosal and autism are not associated in any way.

Yes, there is always the study of disability services data from California -- which seem to be rising among the youngest cohorts of kids, who presumably received little or no mercury because thimerosal was largely removed from childhood shots.

But California is an "ecological study" with problems of its own.

"Although (this) information is often used by media and research entities to develop statistics and draw conclusions, some of these findings may misrepresent the quarterly figures," cautions the website of the California Department of Developmental Services (DDS). "Increases in the number of persons reported from one quarter to the next do not necessarily represent persons who are new to the DDS system."

Even the CDC admits that "there are several limitations" with linking a VSD study design with the California data, Gerberding wrote to Congress, because, among other things, California only counts "persons who were referred to and/or voluntarily entered" the disability system."

It will be interesting to see how the House Committee -- and the mainstream media -- react to this rather breathtaking confession by the CDC, which does seem to want to conduct the best vaccine-autism science possible (see Gerberding's replies to NIEHS recommendations for improving the VSD: CDC officials are currently conducting in- depth follow up studies with VSD patients).

As the waning months of the Bush administration get underway, I can't help but wonder if a little housecleaning might be going on at some of our top health agencies.

(And on that note, please see my blog about the upcoming HHS/FDA/CDC/NIH workshop on autism triggers in mitochondrial disorders (ie, Hannah Poling and her five vaccines).