New HIV infections and diagnoses numbers are out this week from the CDC, and media outlets are fumbling with the numbers. Here's the headline from Time, which was typical:
CDC: 20% of Gay Men Are HIV-Positive, but Nearly Half Don't Know It
Really? How did the CDC even study "gay men," since we're a population that's not all in one place and is hard to define (we still don't have a good estimate of how many America Americans identify as gay because of the slipperiness around identity and the fact that not everyone will tell a pollster their sexuality). If one were to look at the study, which is linked at the bottom of the Time piece so one assumes the writer and editors had access to it, it doesn't make that claim at all.
I won't fault Time's Meredith Melnick for that incorrect headline; she probably didn't write it. And whoever did was probably more interested in shocking than in describing the truth. Melnick gets a little closer in her lede, but she still is several ball parks away:
Recent data from the Centers for Disease Control and Prevention (CDC) show that 1 in 5 sexually active gay and bisexual men in America are HIV-positive but that 44% of them don't know it.
Interesting, since an earlier study this year in the journal Sexually Transmitted Diseases estimated that 11.8% of MSM were seropositive and the CDC estimated in 2006 that 79% of those who were poz knew it. That's a pretty big change in numbers over a short amount of time.
But that's just her first paragraph, where at least she now tells us that the study wasn't just of gay men, but of MSM, which includes bisexual men and men who don't identify as either gay or bi but who have sex with men (whether that makes them gay or bi is entirely up to you, but having sex with a man doesn't automatically make someone gay). But that wasn't the only problem.
Time's Melnick says something different in the second paragraph, without acknowledging at all that her first two paragraphs contradict one another:
More than 8,000 self-identifying gay and bisexual men (or, as the researchers call them, MSM, for men who have sex with men) were tested by CDC workers in the 21 American cities with the highest infection rates. The gay population in Baltimore had the highest rates of HIV infection, at 38%, while Atlanta scored lowest, at 6%.
(A small quibble: 1% of those who participated in the study identified as "straight.")
In her first paragraph, the CDC released data about HIV infection rates among "sexually active gay and bisexual men in America" (emph. mine). In her second paragraph, the CDC released data about HIV infection rates among "MSM... in the 21 American cities with the highest infection rates." That changes things a bit. Contrary to popular belief, men who have sex with men don't live exclusively in 21 urban areas in the United States.
So the title, the first paragraph, and the second paragraph all make different, contradictory statements about the study, with no acknowledgement or explanation from Time. Which is correct? A casual reader wouldn't know, and it's unlikely anyone at Time understands. Many people would probably just read the title, which is the worst of the three, and move on unaware that they've been misled. We're paid to write shocking stories, not report the facts! Melnick and her editors at Time protest.
Anyway, for those who'd like to know, the second paragraph is the one that gets closest, and it's the one least likely to be read. And it's still not good enough.
Here's part of the note that was attached to the study on the CDC's site, which is actually important to understanding the study (Melnick doesn't mention any of this information in her shocking Time piece since it takes a lot of shock out of the story):
The findings in this report are subject to at least four limitations. First, because the survey was administered by an interviewer, positive HIV status might have been underreported during the interview, given the sensitive nature of the topic, thereby inflating estimates of MSM unaware of their infections. Second, 135 MSM who reported being HIV-positive but who had a negative or indeterminate HIV test result were excluded from analysis because of the possibility that they had false-negative NHBS test results; however, including these men as HIV-positive would have yielded a similar overall HIV prevalence (20% compared with 19%). Third, comparisons of the NHBS-MSM datasets collected during 2004--2005 and 2008 should be made cautiously, because this analysis did not control for demographic differences in the samples, which might have influenced the percentages reported. Finally, these findings are limited to men who frequented MSM-identified venues (most of which were bars [45%] and dance clubs [22%]) during the survey period in 21 MSAs with high AIDS prevalence; the results are not representative of all MSM. A lower HIV prevalence (11.8%) has been reported among MSM in the general U.S. population (8).
The CDC acknowledge four limitations to this study, two of which are major limitations that I'll discuss.
In the third limitation, they acknowledge that demographics weren't controlled for. Consider the income of the people tested -- over 30% reported incomes under $20K a year. The Census Bureau only estimated 20% of US household income being below this line in 2009, which isn't a direct comparison since gay households' incomes are less and we're more likely to have a single income for a single person than straight people. On the other hand, they investigated urban areas which have higher costs of living and incomes.
Either way, I can't find comparable data focusing only on the same 21 urban areas and providing information about individual income, and apparently the CDC didn't either since they didn't control for demographics. Since socioeconomic status is one of the major factors that defines HIV infection rates, data that doesn't reflect a representative sample of America's economic reality can't be used to make sweeping statements about "gay and bisexual men in America."
23% of the people studied were "Black, non-hispanic"; 12.4% of America is. 25% of the people in the study were "Hispanic"; about 15% of America identifies as such. Again, perhaps that's representative of these 21 urban areas, but we don't know that and it means that the data isn't representative of "gay and bisexual men in America."
34% of those studied had "no" health insurance, while the un-insurance rate in the US is estimated at 15.4%. This relates specifically to the stories (like the one at Time) that stress how few "gay and bisexual men in America" know they're HIV-positive. If the study focused heavily on the uninsured, then it's more likely to show that people don't know they have a certain virus. This stuff isn't complicated.
The fourth caveat that the CDC lays out is also important: they found their subjects at bars and dance clubs that are "MSM-identified venues." They're missing a lot of people that way since lots of urban MSM (like me) don't frequent gay bars and clubs. They cruise online or in parks. They're in relationships or don't cruise at all. They live far away from any gay bars. They don't like gay bars for a variety of reasons. Maybe they just didn't go to that one bar on the night they were recruiting people for this study. Maybe they didn't want to take part in this study (only 43% of people who were "approached" agreed to participate, making the study self-selecting and not even controlled for basic demographics). Etc.
Of course, none of those caveats made it into the Washington Post story on these data, which ran with this headline and lede:
Study puts HIV rate among gay men at 1 in 5
One in five gay men in the United States has HIV, and almost half of those who carry the virus are unaware that they are infected, according to a new Centers for Disease Control and Prevention study.
The Post's Darryl Fears goes on to call this study "the most comprehensive such research by the CDC," without mentioning any of the caveats the CDC laid out.
Even more frustrating is that the CDC specifically says "comparisons of the NHBS-MSM datasets collected during 2004 -- 2005 and 2008 should be made cautiously," but Fears spends three paragraphs making lazy comparisons between those two exact studies. "[T]he percentage of minorities who tested positive changed dramatically in the three years since the previous study," Fears tells us, not telling us what the CDC know, that the samples in the two studies are different and neither controlled for demographics so comparisons between the two are likely to be inaccurate.
Not that any of that matters. The important thing is that we are all properly shocked into buying newspapers.