Over the past 48 hours, two competing polls in Delaware have provided confusing information to the state's electorate and the nation in general. One telephone poll of 797 likely Delaware voters, conducted by Fairleigh Dickinson University (FDU), October 20 - 26, shows Democrat Chris Coons with a 21% point lead over Republican Christine O'Donnell, 57% to 36% (plus or minus 3.5%). However, another telephone poll of 1,171 likely voters in the state, conducted by Monmouth University's Polling Institute (MU), October 25 - 27, shows Coons with only a 10% point lead, 51% to 41% (plus or minus 2.9%). What are we to make of the competing numbers?
Unfortunately for the public, the two polls are NOT comparable, and without more details it's tough to say for sure which is more accurate. Among the many issues, the interviews were conducted over different time periods, they have different target populations, they define their populations of "likely voters" differently, the questionnaires are different, and most notably, while both polls were done by telephone, live interviewers conducted the FDU poll, and the MU poll used a combination of live (n=402) and automated (n=769) interviews. These very basic features of the polls can lead to drastically different results, and the lesson is "care MORE about the details and LESS about the headlines."
Before I say which poll you should trust, I'll explain some of their key differences, and you can draw your own conclusions before reading mine. Of course, explaining the differences requires digesting some of the dirt in the details.
Geek Warning: Read on if you dare..... (cue Halloween thunder and devious laughter)
When you call and who you call makes a difference
First, the survey designs were different. The FDU poll was conducted over six days targeting landline and cell phone extensions, while the MU poll took three days and targeted only landline phones. Different periods mean people could be using different information to make their judgments. For example, a new effective attack ad, or a public relations error might influence voters' judgments. Also, MU's exclusion of cell phones has important consequences for sampling and coverage error. It's estimated that about 25% of all households are now cell phone only, and research suggests landline only surveys may over-estimate Republican support.
Who's doing the calling makes a difference
Second, the FDU interviews were conducted by live interviewers (by Opinion America Inc.), while the MU poll was conducted using both live interviewers (by Braun Research Inc.) and a computerized interviewer (by SurveyUSA). Different modes produce different interviewing experiences, and sometimes, different levels of bias. While automated interviews are thought to reduced social discomforts about revealing private information, they are also more likely to have interview break-offs (people quitting the interview). Also, since there is no interviewer to confirm the intended interview target, there is always uncertainty about who is actually being polled in automated surveys. Finally, automated interviews are typically shorter in length, and tend to use shorter questions and response categories than live interviews.
The representativeness of the data makes a difference
Third, the FDU data are weighted -- brought in statistical alignment with the actual population proportions -- by age and gender, and the MU data are probably weighted too, but they did not report by what variables (e.g., age, sex, race, county). Data are usually weighted when the sample proportions do not match the population proportions. But survey weights can sometimes inflate (or deflate) numbers leading to inaccurate results. The only way for the public to guage the effects of weighting is to look at the demographic numbers. FDU did not report their descriptive demographic statistics, but thankfully, MU did.
There is something odd about the MU results on partisanship and vote choice. In early October Coons held a 83% point lead among Democrats (91% to 8%), a 10 point lead among Independents (55% to 45%), and held 26% points of support among Republicans; but the most recent poll shows Coons with a 75% point lead among Democrats (85% to 10%) down 8% points, he's now minus 5% points among Independents (42% to 47%), and has 19% Republican support.
This movement is very unlikely in the month before the general election. Numbers simply don't move that much at this stage of the election season; and, especially not that much across all party identifiers in a state where almost half (47%) of all registered voters are Democrat compared to one-third (29%) Republican. Couple this with the fact that Coons has his strongest support in the largest country in the state -- New Castle county, where he has been elected in consecutive terms as County Executive--which is largely comprised of Democrats. Also, it seems O'Donnell has been more controversial over the past two weeks; thus, the evidence points to something potentially off with MU's estimates: at the very least, they seem to over-estimate Republican responses and under-estimate Democrats'.
Well defined populations are essential
Fourth, the two polls use different definitions of "likely voters." FDU's respondents were selected using random digit dialing (RDD) procedures, but it is unclear what questions were used to filter and define a likely voter. The MU poll identified likely voters from a "list of households with voters who cast ballots in at least two of the last four general elections" and say they are either "certain" or "likely" to vote on election day. The aforementioned list (i.e., the sampling frame) was obtained from Aristotle Inc. While the Aristotle list may contain past voters, that doesn't necessarily make it representative of the state's current electorate. Questions abound regarding how "voters" are identified and how the eligible population of voters (e.g., new voters) have changed over the past four elections. In the end, we simply don't know enough about the likely voter populations for either of the polls to judge their accuracy with a high level of certainty.
Question ordering makes a difference.
Fifth, based on the released topline reports and questionnaires, it appears that FDU asked respondents questions about President Obama (favorability rating), the economy (right or wrong track), and the two Senate candidates (favorability), before it asked respondents their preferred Senate candidate. Alternatively, the MU poll appears to ask the Senate candidate preference question first -- or much earlier in the survey than FDU. Sometimes the topline reports do not mirror the questionnaire, but for now they are all we have to go by.
Pollsters are well aware of "question order" effects, but they are hard to estimate across surveys using different designs. In the MU poll, it's uncertain whether their computerized interviews were conducted using the exact same order and question wording as the human conducted interviews. There's nothing wrong with using mixed modes of data collect; however, since MU did not provide the results for the two modes, we can't evaluate the quality of their data.
Reporting standards matter
Just to add one more problem with MU's methodology report, based on sample size alone the margin of error for the live interviewer data is larger than that for the automated interviews, but MU reports one error for the entire study. This is very deceptive and MU should clarify these numbers in accordance with professional standards.
Consistency in polling methods makes a difference
Sixth, both surveys make a statement about the changes that have occurred since their last report; however, only one of them can actually make a fair comparison. In the last MU poll of 790 likely voters, conducted 10/8-11 (MOE +-3.5%), Coons held a 19% point lead, 57% to 38%. This poll was conducted entirely with automated interviewing, but the more recent poll uses both live and automated interviews. In the last FDU poll of 801 likely voters, conducted September 27-October 3 (MOE +- 3.5%), Coons held a 17% point lead, 53% to 36%. Similar to the current poll, the older one was done entirely with human interviewers. Thus, FDU has a consistent method and therefore can more appropriately estimate change, while MU is comparing apples to pomegranates .
And the Winner Is ...
FDU's reliance on the same methodology from earlier polling; the fact that their findings are consistent with other live interviewer telephone surveys of Delaware voters--CNN's poll results from 10/12 show Coons with a 19% point lead; the fact that MU employed a questionable design and reported misleading statistical information; and because some of the estimates for MU's data seem to be off; I would put my money on the FDU poll results.
But, the important lesson is ...
In the end, poll statistics are only as accurate as the methods used to gather the data that created the estimates. More importantly, the media, the public, and elected officials should understand a very important premise: it's not the numbers that matter, it's the inference you make from the numbers. More accurate numbers are good, but they're only confirmed on election day. Both polls come to the same conclusion -- Coons would win if the elected were held "today" (which was a few days ago), so for now just lead with that headline.
Follow David C. Wilson on Twitter: www.twitter.com/dcwilsonphd
_X_ 2008: Change the Guard
___ 2010: Guard the Change
Cooking the booking in polling is nothing new after all, these polls are sanctioned by corporations to fit the narrative they want. Plus, these poll are omitting, cell phone users, and omitting and/or low balling, what they call sub groups, which are minorities. Go vote dems!
http://thehill.com/homenews/campaign/60443-national-polling-firm-battling-back-amid-controversy
http://www.mcclatchydc.com/2010/10/18/102226/beware-of-polls-that-exclude-cell.html
http://ndn.org/blog/2010/10/gallups-likely-voter-model-has-fatal-flaw-ndn-challenges-gallup-revise-or-drop-its-2010
http://www.psbresearch.com/what_we_do/corporate.html
"Weeks prior to the 1948 election, many leading editorial writers and political columnists relied on early Gallup Polls, which predicted Thomas E. Dewey's win over incumbent Harry S. Truman. Truman's strategy was to bypass the press by taking his case to the people in a "whistlestop" campaign. An issue of the early edition of the Chicago Daily Tribune was handed to Truman after the election. The headline declared "Dewey Defeats Truman." Truman upset Dewey despite having the support of only 15 percent of the nation's daily papers."
The best part of it? Well, you see, most of these candidates "loaned," their campaigns money. If O'Donnell, Whitman, etc. win their elections, they can take millions in campaign contributions after the fact from lobbyists to "repay their campaign," ending up not spending any money to get into office-but meaning they will be going from lobbyist to lobbyist with hat in hand, begging for money and happily offering their vote to get it.
When they lose though, their campaign probably won't be able to raise money to "repay," them.
Personally I think we need some way to make self-contributions donations, not loans. Make sure candidates don't enter office in debt and asking for donations to their campaign to restore their personal finances...
It might make someone feel good to "ZING" an uneducated person, but it is far more useful to educate the uneducated. O'Donnell is a tool of billionaires who want to further impoverish the "uneducated"...........let's get that message out....
Nobody can think their way to becoming an O'Donnell supporter; you have to put your brain in neutral and rely totally on your emotional responses, e.g "I'm angry, so I'll vote for a hopelessly unqualified, person who contradicts their position on a daily basis. Makes no sense, but, hey, I'm angry, so that makes it alright."
You want smaller government? How about no more meat inspections? Let the free market simply force some companies out of business after customers start dying of salmonella? Oh wait....conservatives LOVE BIG GOVERNMENT when they are getting a piece of it........then we can watch them WHINE for THEIR "Nanny State".....
And now, the end is near,
And so I face the ballot curtain.
My friends, I'm not clear;
I'll misstate my case of which I'm uncertain.
I've lived a life that's futile -
I've flew each and every highway,
And more, much more than this,
I did it your way.
Regrets? I've had more than a few,
But then again, I forgot to mention.
I did what I had to do
And saw it through without perspiration.
I botched each charted course -
Each careless step along the byway,
And more, much more than this,
I did it your way.
For what is a witch? What has she got?
If not a broom - then she has naught.
To say the madness she truly feels
And not the words of one who spiels.
The record shows I lost the votes
And did it your way.
If they aimed to call only the more Rural areas, then Coons leads by 10%. ID they took a TRUE demographic cut, then the 21% is correct.
In Rural Central & Southern Delaware, O'Donnell is the ideal example of what the conservatives seem to want in this state. It was the Downstaters who got out the vote to allow her to beat Mike Castle. As I drive trhough rural Delaware, I see her signs in front of 2 types of homes, Huge McMansions and the run down rural places with 10 non-running cars in the yard.
All Coons has to do is carry the Vote in New Castle County and the beach resorts areas of Delaware and he will win.
A.I dupont????
LOL F/F!
It might make someone feel good to "ZING" an uneducated person, but it is far more useful to educate the uneducated. O'Donnell is a tool of billionaires who want to further impoverish the "uneducated"...........let's get that message out....
----------------------------
Fanned and faved!!
This explains why Republicans results are skewed and why,IMHO,Democrats may lose seats (typical in off years) the Repubs/Baggers are not going to do as well as they think. If I am proven correct expect to see Repug/Baggers engage in two of their favorite activites...whining and making up "facts" to "support" some victim card scenario
This is the party that preached fiscal discipline and then cut taxes in time of war.
Remember that Republicans consider tax dollars spent on American citizens to be a waste of taxpayer money.
This is the party that still wants to put the torch to Social Security and Medicare.
Remember that Republicans believe in state rights UNLESS a state passes a law that doesn't coincide with their narrow point of view.
This is the party that wanted to add a "Marriage Protection Amendment" to the U.S. Constitution.
Remember how Republican ears are deaf to the screams of the people yet hear the whispers of Big Business.
This is a party that, given a choice between Abraham Lincoln and Ronald Reagan, would choose Ronald Reagan in a heartbeat.
http://www.youtube.com/watch?v=QlBOv8m_Xa8
On November 2, 2010 - REMEMBER!