I have just managed to read a copy of Brian Strom's written, public submission to the House Committee on Oversight and Government Reform, which wasn't completed in time for the oversight hearings on the diabetes drug Avandia on June 6, and so didn't appear in any of the media accounts of the hearing. Dr. Strom is Chair and Professor of Biostatistics and Epidemiology and Professor of Medicine and Pharmacology at the University of Pennsylvania School of Medicine, and what he says makes for interesting reading. Typically, the medical world engages in toneless, clinical and jargon-larded criticism that makes it a snooze fest for the rest of us.
But Strom's submission is readable, and, at least literally, polite -- he describes the Nissen and Wolski study, which purported to show an increased risk of heart attack associated with Avandia, as science that was conducted "quickly" rather than "ignorantly." At the same time Strom's criticism is deadly. The meta-analysis, he writes, is "very weak," adding that most of the studies it looked at were unpublished, which is to say that they had not been peer-reviewed.
While this avoids the issue of publication bias," writes Strom, "it raises other important issues, in that the underlying studies in this meta-analysis were never subjected to the usual criteria of scientific peer review. This leaves uncertain the quality of the underlying information.
Strom also notes that:
Inherent in the technique of meta-analysis is the assumption that the studies are all giving the same answer. Yet, these studies are extremely different from one another, with different patient populations, and even different comparison groups. Some of the studies compared Avandia to placebo, while others compared Avandia to various other antidiabetic drugs.
The largest study, by far, considered in the meta-analysis did not even include patients with diabetes. Combining all of these studies into a single analysis is quite questionable.
And he points out that as the studies selected by Nissen were not designed to study Myocardial Infarction (heart attack) the patients may well, on previous experience with such studies, have not experienced it at all. Strom concludes that despite all the above limitatons, Nissen and Wolski were still only able to arrive at "borderline statistical significance" and that this means the findings could be the result of random error.
In summary...it is clear that the study raises an important question. However, it is exceedingly far from providing an answer to that question. Indeed, this is a question that had been raised well before this paper was published. The two other studies available so far do not support this finding, although they too are not definitive. The best way to address such a question is a large outcomes-oriented randomized clinical trial, and one is underway, i.e., RECORD. Ultimately, the hypothesis identified in the Nissen paper may indeed be proven to be correct, but it may not be.
As I noted, however, because of the publication of Nissen's study, the way its author hyped it to the media (comparing Avandia to 9/11), and the way NEJM buttressed it with several strongly worded editorials, the RECORD trial may be now be in jeopardy from patients dropping out in a panic over their perceived risk.
This analysis also makes the decision to file the first Avandia lawsuit over the fatal heart attack of a 60-year old man in Texas on May 21 something of a challenge.