Our friends over at the Hotline
OnCall looked at the results of the latest CBS/NY Times survey and raised intriguing
Either early polls are fickle or Sen. John
McCain is improving his standing vis-a-vis Rudy Giuliani,
In the latest
New York Times poll, Giuliani leads McCain by nine points, 43% to 34%. Last
month, Giuliani led 50% to 29%.
Both the February and March surveys involved a hypothetical
two-way match-up between Giuliani and McCain (Q49),
and the decline in Giuliani's support appears to be just barely statistically
significant. For those wondering (as I did), surveys interviewed 314 registered
Republicans voters who say they are likely to vote in Republican primary in February and 603 in March (the most recent survey involved an oversample of
Republicans; unweighted counts are at the end of the CBS releases here and here).
Given the sample sizes, the seven point drop for Giuliani
(from 50% to 43%) is just barely statistically significant at a 95% confidence level,
although the five point increase for McCain (29% to 34%) is not.
However, there are some very good reasons for caution here. First,
when I say "barely" I mean the Giuliani decline would not be significant if we needed to be 96% confident. And the precision of my calculation requires a
statistical assumption (simple random sampling) that is iffy when pollsters
weight their data (as this and virtually all other public polls do).
Next, remember that four other surveys conducted over the
last 10 days - including the just released CNN/ORC
survey - show no comparable Giuliani decline. In fact, while the changes on
any individual poll may have been within sampling error, all four show a slightly
larger Giuliani lead (see also the trend in evidence on our Presidential tracking chart).
Keep in mind, of course, that the CBS/NYT survey involved a
hypothetical two-way Giuliani-McCain
match-up, while the others asked respondents to choose among all of the likely Republican candidates.
Notice that the CBS/NYT results are better for both Giuliani and McCain - the
perfectly logical result of offering just two choices rather than roughly ten. Perhaps
the Giuliani decline occurred mostly among those who only opted for Giuliani
(in February) because their first choice was not an option.
Consider finally the issue of the "oversample" of
Republicans on the most recent survey interviewed by redialing households "at
which a self-identified Republican had been interviewed in [CBS/NYT] polls over
the last seven months." Of course, the pollsters can compare the oversample to
Republicans in the base sample to see if the former was more hostile to
Giuliani. In response to my email query, CBS Survey Director Kathleen Frankovic
confirms that she saw "no indication that the oversample was hostile to
When it comes to methodology, the CBS/New York Times surveys are top notch. But yes, early or late, even
the best polls can produce "fickle" results. So given all of the above, readers
should be cautious about making too much of the observed Giuliani decline until
UPDATE (3/14): Another possibility to consider is that CBS chooses it's likely primary voters a bit differently than the other national polls. For one, they actually attempt to identify likely primary voters, something not done by the other national polls (as far as I know). See this post today for more details. If the CBS/NYT poll is looking at a narrower population than other polls, they may be picking up emerging trends among true primary voters more readily.