Novak Calls It a Push Poll

03/19/2007 05:32 pm ET | Updated May 25, 2011

Before I leave the topic of "push polls"
and the indiscriminate way reporters and pundits use that label, I want to
mention a prominent example that appeared earlier this month. Conservative
commentator Robert Novak led off his March 1 column
with this reference to a survey of Republicans in Iowa:

York-based political consultant Kieran Mahoney's statewide survey of probable
Republican participants in the 2008 Iowa presidential caucuses shows this
support for the "big three" GOP candidates: John McCain, 20.5
percent; Rudy Giuliani, 16.3 percent; Mitt Romney, 3.5 percent. Astonishingly,
they all trail James Gilmore, the former governor of Virginia, with 31 percent.

How could
that be? Because it was not a legitimate survey, but a "push poll."
That normally is a clandestine effort to rig a poll by telling respondents
negative things about various candidates. Mahoney makes no secret that his
voter sample was told of liberal deviations by McCain, Giuliani and Romney, and
of true-blue conservatism by Gilmore (Mahoney's client)

"Illegitimate" would be a clearly appropriate term had the sponsors
presented their results as a fair reading of current caucus preferences in Iowa. However, Novak tells us that Gilmore's
consultant "made no secret" that the survey question relayed information about
the "liberal deviations" of the other Republicans and "true blue conservatism"
of Gilmore. So while questions may remain about the poll's fairness, the
release itself is not quite as deceptive as Novak seems to imply.

Either way, the results as described do not add to a "push poll." That more nefarious
dirty trick is a truly "clandestine effort" to communicate to a mass electorate
with telephone calls made under the guise
of a public opinion poll (columnist Stu Rothernberg prefers
the term "advocacy call"). Rather, in this case, Gilmore's consultant
apparently conducted a legitimate poll that tested how a random sample of
likely Iowa
caucus goers would react to a set of messages that were (apparently) highly
favorable to Jim Gilmore.

Again, even if his terminology was sloppy, Novak was right to distinguish
this result from an attempt to measure current
vote preference. In this case, the Gilmore survey used what some call a "push
question" (others an "informed vote") to see how specific messages would move
the Iowa Republican electorate. Such exercises are common, legitimate tools used
by campaign pollsters to gauge the way voters will react to new information
received during the campaign. You can see many examples in the surveys
regularly released by the Democracy
project of Democratic pollster Stan Greenberg.

Now, describing a poll as "message testing" does not let the pollster off the
hook. In some cases, pollsters work to make these questions as fair and
even-handed as possible. In others, their efforts may be fairly characterized
as "rigged" to produce results that work in one candidate's favor. And
sometimes - in a circumstance that brings joy to every campaign pollster - they
can do both at the same time.

So was the Gilmore question fair? It is hard to tell in this case,** because
neither Kieran Mahoney nor the Gilmore campaign are willing to share the full text
of the question with anyone other than Robert Novak. I spoke with Kieran Mohoney
today, and he explained that he let Novak see the results and the verbatim text
because he believed the columnist would see his characterizations of the
candidates as fair. Mohoney believes that Novak's review "constitutes as much
public validation as I'm interested in at this time," and politely declined to
release the text.

All of which brings me to three pieces of advice:

1) Be highly skeptical of results from an "informed vote" (or any other form
of "message testing") that does not include
the full, verbatim text of the questions. We have no way to judge the fairness
of the questions without reading their text. If you are a reporter, do not even
think about reporting such results unless you can see the full text, and
(ideally) point your readers to it as well.

2) A skeptical and critical read remains in order even when you do have access to the full text. Ask
yourself, are the candidate descriptions balanced? Do they provide only
positive information about one candidate and negative facts about the others? Do
the descriptions leave out important points that candidates will emphasize? Do
they give some issues an unlikely prominence? Remember, even a perfectly fair
"push question" attempts to predict the information flow of a real campaign,
and that is not an easy thing to do.

3) If you write about an unfair or biased result from a message testing
poll, please, please refrain from calling it a "push poll." The English
language leaves you many fine terms - negative, unfair, untrue, distorted,
biased, slanted and, yes, even "rigged" - that will describe an offensive poll with
far more accuracy.

**We can evaluate one aspect of this result, by reading between the lines a
bit. Novak's description implies a skew against other conservative alternatives to Giuliani, McCain and Romney,
such as Sam Brownback and Mike Huckabee, because their credentials were
apparently not described.