Not quite two weeks ago, I wrote about a survey of Indiana's 8th Congressional District conducted by for the Evansville Courier & Press by Indiana State University's Sociology and Research Lab. Of the registered voters that responded to to that survey, 63.5% were women, and along with Bill Cullo at Crosstabs.org I wondered whether that unusually high proportion of women might have skewed the results. I wrote that the editors at the Courier & Press "owe their readers some indication of how this very unusual result may have affected the results." About a week ago, they did just that.
A follow-up article explained that Thomas L. Steiger, the sociology professor who conducted the poll, "didn't see the need to adjust for gender in this case" but "for the sake of discussion . . . did a calculation that gave more weight to men." The result:
There was only a fractional change in the spread between Hostettler and Ellsworth.
"There is no evidence of sex bias in this sample related to candidate preference," Steiger said.
"I do disagree with altering poll results. What is important is to test what is being asserted ... and there is no statistical evidence from this sample that females" bias the survey's results in Ellsworth's favor.
In fact, women and men were nearly equal in how they responded to the question about which candidate they preferred: 48.2 percent of men favored Ellsworth, and nearly the same percentage of women - 47.1 percent - favored him, too.
The author went back to Bill Cullo,who agreed with Steiger:
Cullo, however, now agrees that gender had a very minimal effect on this poll.
"I didn't have a problem with Hostettler trailing," Cullo said, "but to not adjust for gender calls things into question. It would have been wise to go ahead and weight the data for gender, though. But it's not nearly the disparity I thought it would have to be. I thought it would be a much more stark contrast."
In fact, when Cullo learned of additional polling data from the Courier & Press, he said Hostettler could be in bigger trouble.
See the full article for details, and lets give the Courier & Press credit for trying to shed more light on this subject.
And as long as we are on this subject, I want to clarify something that confused at least one valued reader about my initial post. I wrote:
Most media pollsters begin with a sample of all adults, and weight the adult sample to match the highly reliable demographic estimates from the U.S. Census. They then select a pool of registered or likely voters from the larger adult sample, allowing the demographics of the sub-sample to vary.
I did not mean to imply that the process of selecting registered for likely voters involves a second round of random sampling. Pollsters simply select the subgroup of interest (self identified registered voters, or voters that they classify as "likely") from the larger sample. The process is analogous to selecting any other subgroup (women, 18-30 year olds, union members, etc.).
And I should again make clear that weighting or adjusting a sample of all adults by demographics like gender and age is not controversial among media and political pollsters, because, as I wrote in the first post, we can base those adjustments on highly reliable U.S. Census estimates of the adult population. The practice of seperately weighting the subgroup of registered or likely voters -- the issue in the Courier & Press survey -- is more controversial, because the demographics of those populations vary slightly from election to election, and estimates are less reliable.
My point was that campaign pollsters are typically willing to make educated guesses about the demographics of voters. Media pollsters, including ISU's Steiger, are far less willing.