[Margie Omero is President of Momentum Analysis, a Democratic polling firm based in Washington, DC.]
Wednesday, Politico told the story of a single poll number getting mistakenly pushed around through blogs and talking points. Republican talkers from Rep. Putnam to Matt Drudge to Freedom Watch announced a "single-digit" congressional approval rating. Their proof was a Rasmussen poll that asked respondents to rate Congress using 4-point job scale: excellent, good, fair, or poor. Typically, one would call this a "job rating" and combine excellent/good to be "positive" and the fair/poor to be "negative." In this particular poll, Congress did receive a nine (9%) positive rating.
What Congress did not receive in that poll was a single-digit "approval rating." That is a different type of standard question altogether. An approval question usually reads "do you approve or disapprove of the job Congress [or whomever else] is doing?" While some use a 4-way approval rating, collapsed into two categories, most have only two categories (besides an "unsure" option). And all use the word "approval" as opposed to an entirely different set of words. Even the most cursory scan of public results demonstrates that a collapsed 4-point job rating scale will typically yield a smaller positive rating than will an either a 4-way collapsed or 2-way approval question.
It's not that one type of question is better than the other. But the shorthands that have emerged for particular questions mean something to pollsters and poll-watchers. To avoid confusion, it's best to just make sure you're comparing apples to apples, and using the clearest terms available.
Just as some reminders, here are some common other wording specifics to be on the lookout for when comparing across polls. (If you haven't already, also check out the pollster.com FAQ.)
Party ID vs. party registration: Definitely not the same thing. Identification is self-reported, and subject to national trends, local press, and respondent whims. Party registration requires some interaction with the state, and varies massively from state-to-state. In many states, voter declares party affiliation when registering to vote. In some states, like Ohio, "registration" refers to which party's primary ballot was recently pulled, rather than requiring a voter to declare their party in advance. Other states, like Missouri, have no party registration at all. In national polls, "party" means identification. But in state or Congressional district polls, the pollster should specify.
The "Re-elect:" Many pollsters ask a "re-elect" question about an incumbent, which includes only the incumbent and no challenger names. An example, "Would you vote to re-elect Mystery Pollster, would you consider someone else, or would you vote to replace Mystery Pollster?" The question wording varies (such as the SC public poll here), and some pollsters use a 2-way question (re-elect or not). Many just look at the response for re-elect and ignore the rest. But the "replace" can also be a useful figure, as we note in our own poll for Congressional candidate Victoria Wulsin (OH-2), which shows the Republican incumbent's "replace" as high as her re-elect.
Leaners: Typically respondents initially undecided in a vote are asked a follow-up, something like, "Well, if the election were held today and you had to decide, toward which candidate do you lean?" Net support for each candidate would then include leaners. But it doesn't have to. Leaners can be included in the undecided. A good polling memo or story should simply specify.
Public disclosure of calling methodology and weighting schemes are of course important, particularly with the closely followed national media polls. But that information is not always available, or easy for the average poll reader to decipher. In many cases, paying attention to wording differences, and asking pollsters for their question language can minimize reporting gaffes.
How will Trump’s administration impact you? Learn more