WSJ-NBC Poll cuts corners (again)
Here's how to evaluate a silly national poll showing Cruz ahead
"Ted Cruz Overtakes Donald Trump in Latest Republican Presidential Poll" - that's the headline of the Wall Street Journal, citing its joint national poll with NBC, the results released yesterday. The subhead is, "Texas senator leads businessman 28 to 26 percent nationally." Really?
The national survey was completed February 14-16, entirely after the South Carolina debate on Saturday night (February 13) when Donald Trump went, as they say, "over the top" in accusing former President George W. Bush of being a liar about the Iraq War. Perhaps that hurt Trump's national standing among Republicans. And while this poll was in the field, George W. Bush was in South Carolina campaigning for his brother. Possibly the national coverage of the former president emerging from hibernation to campaign eloquently for his younger brother did help Jeb (a little?), but somehow, did it also in a big way nationally hurt Donald Trump and help Ted Cruz? Based on my decades of expertise, I would doubt that.
This WSJ-NBC survey is good news for Sen. Cruz, and the findings could be correct (though they are not, as i shall note). After all, Ted Cruz is the most credible conservative in the race. He has run a disciplined campaign. He has raised a lot of money. His campaign has produced some creative commercials. Besides their originality, perhaps they even get him votes. Maybe. And consider that Republicans may be uniting behind Cruz, against Trump. But, as I explain, not yet!
If Trump has dropped nationally from a solid first place lead to a close second place behind Cruz, we would expect to find evidence of that movement in individual states. After all, this is a macro-shift, even seismic.
So, why not look into a state with a lot of polling - South Carolina, which holds its Republican primary this Saturday (February 20). Cruz is a "values conservative" solid on social issues, so he should be doing well among South Carolina Republicans, of whom nearly two-thirds are self-identified evangelicals. In other words, you would expect Cruz to be doing better in South Carolina than nationally. Whatever is happening nationally would be magnified in South Carolina, where voters are more focused,
Regardless of exactly what the numbers show in South Carolina, we should be seeing a trend of Trump going down and Cruz going up. We should see confirming evidence of major national movement. There are no less than a half-dozen super recent SC polls, all except one ending on Tuesday, February 16, with much of the polling after the February 13 debate. The six polls are Bloomberg (2/13-2/16), Monmouth (2/14-2/16), South Carolina House Republicans (2/16), Emerson (2/15-2/16), ARG(2/14-2/16), and PPP (2/14-2/15). These polls show Trump between 33 and 36 percent, averaging 34.5 percent. Cruz is anywhere from 14 to 20 percent, averaging 17.3 percent. Is it likely that Cruz is narrowly ahead nationally, but Trump is ahead 2-to-1 in South Carolina? Assume extremes: Give Cruz his highest polling number, 20, and Trump his lowest, 33. How could Cruz be ahead nationally, and still be behind Trump by 13 points in South Carolina?
Of the six surveys, the most recent South Carolina survey was completed in one night (Feb. 16). Polling in a single night can be volatile and unreliable. That said, this SC House GOP survey included 2400 likely voters (+/- 2 percent), about as many as in the other five surveys combined. It showed Trump 35, Cruz 16.
Back to the WSJ-NBC national poll: it contrasts sharply with the findings of a Quinnipac national poll. In fairness, the Quinnipiac poll was 2/10 to 2/15, while the WSJ-NBC poll was slightly more recent (2/14-2/16.). It is not likely, but certainly possible, that there was massive national movement during the last two days of Quinnipiac sampling, and particularly on 2/16 when Quinnipiac was out of the field, and WSJ-NBC was still interviewing. But consider that Quinnipiac sampled 600, which in its six days was similar in the number of daily interviews to NBC/WSJ 's 400 for three days. The movement on 2/16 would have to be, on that one day, YOU-U-GE for Cruz, and against Trump, somehow reliably measured by that one day's sample of perhaps 133 (+/- 9%).
Two more points: USA Today's national poll (2/11-2/15) showed Trump 35 percent, Cruz 20 percent or numbers similar to Quinnipiac. This morning CBS News released its national survey (2/12-2/16). Its 581 completed interviews began two days earlier than WSJ/NBC and ended on the same day. But CBS also shows what I'll call the consensus numbers -- Trump 35 percent, Cruz 18 percent. If there were massive movement to Cruz in the last couple of days when the CBS poll was in the field, the CBS summary analysis would qualify the findings by noting such a shift.
The pundits (most of whom have never run a campaign) and the CNN-FOX "strategists" (most of whom have never formulated a successful campaign strategy) and the network commentators (most of whom lack any quantitative aptitude) explain all this madness by citing the much-referenced, deeply understood ("plus or minus") "margin of error." They subtract this "sampling error" from one candidate and add it to another candidate. This "statistical margin of error" assumes (which they do not realize) NO non-sampling error, that is, no methodological errors, and thus assumes the proper definition of the voting universe, a correct derivation of the sample, calling distribution over certain days or day-parts, adjustment for cell phones, adequate callbacks, perfect interviewing, no questionnaire bias, and a bunch of other factors that are not statistical in nature.
Polling would be worthless if you assumed sampling error describes the extremes of each survey. Why take polls if, as a matter of course, they could be regularly off by the extreme error range? In reality, in a professional survey, non-sampling error is -- or should be -- nearly nil, and sampling error measures a likely maximum statistical error. But the results are more likely correct than off by the full range of theoretical error, or off by one or two percent, rather than a statistical error range of five or six percent.
In theory, one out of twenty times (that is, with a 95 percent confidence interval common in such polling), the results could be an outlier off by more then the margin of error. I polled for more than three decades and almost never had an outlier. In all that time, my surveys were either on the mark or very close. It is not a matter of statistics, it is a matter of dong it right and controlling for NON-sampling error. The WSJ-NBC poll out yesterday is not an outlier, it's just plain WRONG.
Many of the most publicized national surveys cut corners to save money. Some polls may be sloppy. Interviewers may not be supervised. The screening criteria to include respondents may be off. All this is not "sampling" or 'statistical" error, or "margin of error." If you define the universe of voters incorrectly, it doesn't matter if you draw the sample "correctly" from the WRONG universe.
I discussed the WSJ-NBC poll last year.
Its national sample is small and of "adults," of whom a larger than normal percentage implausibly claim to be voters; the "Republicans" are not drawn from a registered voter database but are self-identified and unverified; this Republican primary sample even includes others (!) who claim they will vote in a Republican primary (even if they legally cannot in their state). In the past, the WSJ-NBC survey arguably has not allowed properly for voter history, voter propensity, or an updated turnout model.
I'm not talking about South Carolina or any individual state, or movement among or between candidates other than Trump or Cruz. We're talking about the national numbers for these two candidates. Nor am I talking about whatever happens, one way or another, in response to the Pope's attack on Donald Trump.
At some point, Ted Cruz could lead Donald Trump nationally. Who can predict the future with absolute certainty? But Cruz is not leading Trump nationally, right now.
An earlier version of this appeared in THE DAILY CALLER.