Did FBI Director Comey swing the presidential election to Donald Trump with his letter to Congress about additional Hillary Clinton emails on the laptop of Anthony Weiner in late October 2016? The answer to that question has been the focus of a back and forth discussion between Nates Cohn and Silver in the past week.
Nate Silver argues that a story that was at the top of the news for six of the seven days following the October 28 letter clearly had an impact on Clinton’s numbers. He makes the case for either a large or small impact, and leans personally to a small one, which dropped her lead in swing states from 4.5 points to just 1.7 points a couple days before the election.
Nate Cohn counters that he believes that the lag in time between when a poll fields and is published indicates that the negative slide in Clinton’s lead had already occurred before the release of the Comey letter. He does not conclude that the Comey letter was a total wash, but he states that there isn’t enough data from before and after the letter to definitively show its effect.
Trendency Research does possess data from before and after the Comey letter, as our panels of voters are continuously taking our surveys and feeding data into our system. Our data shows that both Nates are right. Indeed, Trump was already improving his standing among swing state voters, so he might have caught up to her regardless of the letter. At the same time, the letter had a clear impact on the race and lowered Clinton’s chances of winning.
As a brief background for the uninitiated, Trendency Research convened panels of voters in various swing states for the 2016 election cycle. The power of these panels lies within Trendency itself, which is an online survey platform that incorporates predictive analytic algorithms. While traditional survey research takes a snapshot in time, respondents on Trendency take our surveys at frequent intervals. The continuous stream of data from our panels and our proprietary algorithms provide incisive and timely analysis to show exactly when attitudes shift and by how much. For a longer look at the 2016 Trendency results, click here for our wrap-up memo.
These panelists log in to our system and take short surveys that include questions on the horserace. For electoral questions, we move away from a binary view of the election (such as, “Are you voting for Trump or Clinton?”) and instead ask respondents to allocate their vote out of 100 percent. So, one day a voter might say they are an 80% Clinton voter, 20% Trump, and a week later might be 65% Clinton, 35% Trump. In traditional polling, that same respondent would most likely tell the caller they are supporting Clinton each time. But we believe that this movement away from Clinton is significant and needs to be factored into analysis. Furthermore, Trendency can compare the relative strength of support for each candidate, adding more context to the simple topline number.
Swing State Averages
In the period between the party conventions and the election, we ran panels in 5 key swing states that Clinton ended up losing: Florida, North Carolina, Pennsylvania, Ohio, and Wisconsin. There are two ways to look at the data: first, as an average across the five states, and then secondly as individual states, which allows us to dive a little deeper into a couple of examples.
When analyzing results on Trendency, we often break out the results using two metrics: the Commitment Index and the Rejection Index. The Commitment Index measures the percentage of people who are fully committed to a certain candidate. These people, while able to change their opinion, are unlikely to do so. The Rejection Index flips that around and breaks out the people who will most likely never support a certain candidate.
The above graph plots the average Commitment Index across the 5 swing states from mid-August up through Election Day and includes the timing of several key events. As we can see, Clinton’s Commitment Index started increasing right before the 1st debate and continued its upward trend right until the Comey letter. Trump’s lacking performance in the 1st debate certainly affected his standing, but between the 2nd and 3rd debates - right around the time the Access Hollywood video was released - his Commitment Index increased. For everyone who thought that would be the end of his campaign, sorry to hit you with that unwelcome news.
Importantly, Trump had already caught Clinton before the Comey letter was released. Clinton’s increase on the scale had been moving at a slower and slower pace, but she clearly plateaued after the letter while Trump received a bounce several days later. Trump’s growth also slowed for a couple days, but given the previous days leading up to the letter, it is certainly reasonable to assume that Trump would have led Clinton even if the letter never happened.
Flipping to the average Rejection Index, there is a more discernible impact of the Comey letter. The 1st debate quite clearly moved each candidate’s numbers, but then Clinton’s Rejection Index stabilized right at 50 percent - still not the best number, but better than her opponent’s. After holding at 50 percent for about 3 weeks, Clinton’s Rejection Index jumped to 52 percent right after the letter. At the same time, while Trump’s Rejection Index did drop right after the letter, it didn’t stay down, and both candidates ended up tied with 52% of voters saying they were unlikely to support either candidate.
While we can never know definitively what would have happened had the letter not been released in terms of the Rejection Indices, the fact that both candidates had plateaued in the week before the letter suggests it had a clear effect. This is especially true for Clinton, whose Rejection Index had not moved since about the 3rd of October.
Looking at the data a different way, we can compare the net difference between the Commitment and Rejection Indices. Again, we note that the debates, especially the 1st one, were quite beneficial to Clinton. In fact, she improved her standing all the way through the 3rd debate. Once the debates moved to the back burner, as they did during the rapidly changing news cycles of the 2016 campaign, Clinton’s improvement ceased. Right when the Comey letter was released, her net index dipped and continued going down right until November 7th.
While it might be tempting to conclude that Comey was decisive, it is important to note that Trump’s numbers began to increase as well about a week before the letter. Since we can’t re-run the election without the letter, we will never know if Trump would have caught Clinton regardless. That being said, we should point out that even with the letter, his net difference never reached the level Clinton was at prior to October 28th.
Individual State Analysis
Just as two seemingly competing narratives exist to explain the importance of the Comey letter across five swing states, a closer examination of two individual states, Wisconsin and Florida, shows a divergent impact.
Wisconsin
The impact of the Comey letter for Clinton appears to be strong and decisive here. That being said, Clinton was not guaranteed to have won the state even without the letter. As the chart below shows, Clinton’s substantial lead in the Badger State had already greatly diminished before October 28.
We developed the Aggregate Performance Score (APS) to show the overall state of the race across several important measures including the Commitment and Rejection Indices. We have found that any score between -5 and +5 is essentially in the toss-up category, while anything outside that range leans more definitively towards one candidate.
In Wisconsin, Clinton held a solid lead in August and through mid-September. From that point on, however, Trump made large gains. While the 1st debate stopped the bleeding, Clinton’s downward trajectory continued, albeit at slower pace. From the end of the final debate through the Comey letter, Clinton held a slight lead, but barely outside the toss-up category. It was not nearly as safe as some public polling and the Clinton campaign assumed it to be.
While the Comey letter appears to have had mixed impact when looking at all five states, in Wisconsin, the letter clearly made its mark. The APS was stable just at the margins of the toss-up area and then dropped precipitously in favor of Trump. By Election Day, the score had moved clear of the toss-up range and into Trump’s camp.
As we have mentioned before, we can’t examine every counterfactual in this election. It is quite possible that Clinton could have held her very slight advantage in Wisconsin without the letter, but it is also true that the overall trend was in Trump’s favor. Without any method to test this hypothetical situation, we must look at the data as we have it. In Wisconsin, the Comey letter appears to have immediately moved the needle and pushed Trump to victory.
Florida
While the Comey letter had an impact on Wisconsin, Trump does not appear to have benefited in Florida from its release:
The Trendency data always had Florida as a state that leaned toward Trump, despite what public polling might have shown. As with many states, the debates were a positive force for Clinton, moving the state to just outside of the toss-up range. Once the debates ended, Trump regained some of his lead, which held steady even after the Comey letter. Indeed, his lead stalled and even decreased in the final days before the election, something unlikely to be caused by the Comey letter. Florida was a lean Trump state on October 27 and ended that way on Election Day.
So, what can we take away from the disparate impacts of the letter? When examining all five swing states, it does appear that Comey increased the number of people who rejected voting for Clinton, making it much harder for her to pull off a victory. At the same time, the percentage of voters who were committed to voting for Trump was already on the rise before the letter. While it is hard to say that the letter alone was the root cause of her loss, it is clear that it did not help her. The combination of Trump’s existing rise with committed voters combined with the Comey letter doomed her chances and ultimately make both Cohn and Silver correct in their analysis according to our data.
