Given the sheer volume of polling data made public every day, and given that there sometimes seems to be a lot of variation in what the polls say about individual races, it is not surprising to hear doubts expressed about the overall quality of political polling. Charlie Cook, for instance recently expressed a preference for candidate or party-sponsored polls and a skepticism of "independent" polling, even referring to academic polls and those sponsored by local media as "dime store junk." At the same time, though, polls are pretty useful in predicting outcomes. In earlier posts, I've shown that both House and Senate outcomes track very closely with aggregated polls, and Nate Silver also touts the predictive power of polls, using them as a central component in his forecasting models.
My point in this post is not to sort out which types of polls are more accurate than others, though that might come in a later post. Instead, I'm interested in looking at the average rate of polling error. Although there are a lot of things one could look at to judge the quality of a poll (question wording, sampling method, etc.), my guess is that most people focus on the bottom--how close the poll finding is to the election outcome--when judging trial-heat polls. For better or worse, this is likely to be the case. Toward that end, I examine the error in Senate, House, and gubernatorial polls taken in the last fifteen days of the 2006 and 2008 campaigns.
The histograms below show the distribution of polling error across offices and years. In all of the figures, I compare the Democratic percent of the two-party vote result to the Democratic percent of the two-party poll result. Data for all of these graphs come from
pollster.com.
To be sure, there are differences across years and across level of office, but the overall picture is not one of wildly inaccurate polls. Some of the key findings are:
- The statewide polls in Senate and gubernatorial elections are generally more accurate than district-level House polls. House polls in 2008 had the highest error rate.
- Most polls (90% of Senate polls, 88% of gubernatorial polls, and 81% of house polls) are within five points of the actual outcome.
- Overall, the election outcomes fall within margin of error of individual polls 79% of the time. Theoretically, this should happen 95% of the time.
One thing to bear in mind, though, when looking at these error patterns, is that the deck is stacked somewhat against at least some of the polls. I used a fifteen day window to make sure I had a large number of polls for each year. One consequence of this is that many of the polls are being compare to an outcome that occured several days after they are conducted. Assuming that there is at least some real movement in vote intention during this time period, one might expect a greater discrepancy between polls and votes than what is reported above.
Given the sheer volume of polling data made public every day, and given that there sometimes seems to be a lot of variation in what the polls say about individual races, it is not surprising to hear d...
Given the sheer volume of polling data made public every day, and given that there sometimes seems to be a lot of variation in what the polls say about individual races, it is not surprising to hear d...
I live in NE FL and a registered Democrat and only recieved one call during the primaries. I was at a jewelry party last night and the home owner is a registered Repub and the phone would not stop ringing!!!! from all the polling co. What a joke!!!!! The news media should stop reporting this gar bage!!!
1 .most people don't understand them
2. they are reported dubiously by the media.
3. polls depend on land phone lines and the younger voters (and often the older ones!) don't have land lines.
There is one young man that seems to read the polls right....and dang, I can't remember his name....he is on MSNBC often. Black hair, black glasses, and is nervous on camera so, doesn't smile much.... dang.... help! I know one of you know his name...
Did you even read this article?
The problems with today's polls are in two main areas, special interest polling that is intended to skew results, and improper interpretation of results.
1) Special interest - When a poll is commissioned and paid for by a special interest, it is usually to "prove" a point. One example was a poll by Zogby for a conservative radio host. The supposed purpose was to "test knowledge" of Obama voters. In reality, the questions were selectively chosen and biased to present only the case the customer wanted to show. These types of polls offer little or no useful information. They are intended to influence public opinion, not gauge it.
2) Improper interpretation - I have seen many instances where polling companies draw conclusions that are not supported from the data. One example was the so-called "ground zero mosque". I have seen interpretations that indicate certain numbers of people oppose the mosque. However, many of the polls did not gauge the level of opposition. There were plenty of people who only thought it was a bad idea, not that the construction should be blocked. This kind of misinterpretation can be a huge problem, because so many people do not take the time to look at the actual numbers. They read a headline and a little bit of text and think the numbers say something that they really don'.t
The author looked at several polls to examine the trends. He did not use "a polling company". He used math.
I think my point is made?
Second question: how would you know it was competitive or not unless you saw a the poll? And races do change over time. Competitive races become runaways, and vice versa.
And as to the second, if it's not going to be close, you know.
The pendulum swings both ways, sweetheart.
At the end of the day, there are more reasons to vote than "I want my candidate to win." What about civic duty? Exercising your hard-earned freedom to express your opinion, even if it not in line with the majority view?
Not to mention the fact that closeness of elections impact fundraising and candidate quality in subsequent cycles.
THERE ARE PLENTY OF REASONS TO VOTE no matter what the polls say.