Nate Silver's rational approach to politics seems to provoke highly irrational responses.
At Slate, Daniel Engber writes that Silver, author of the FiveThirtyEight blog at The New York Times, "appears to have hit the mark in every state -- a perfect 50 green M&Ms for accuracy." Engber links to a map of Silver's predictions versus a map of the results. Impressive, right? But Engber can't leave it at that. He notes that the averaged state-wide polls said much the same thing: "Nate Silver didn't nail it; the pollsters did."
A little later, however, Engber hints that he knows better. "To be fair, the art of averaging isn't simple," he writes. Ah, there's the rub. Engber acknowledged that "Silver judges which polls to put into his analysis and then he weighs them according to his perception of their quality." Again, not quite right. What Silver does is not an "art"; it's mathematical modeling. And Silver doesn't make judgments the way pundits do, trusting their gut, their instincts, or their experience. He assesses polls based on how they performed historically, among other things. Bad data can produce bad results; no one would disagree. So Silver uses statistical and other techniques to scrub his data, something scientists in many disciplines have been doing for decades.
What Silver is doing is fundamentally different from what politicians, political analysts, pundits, and commentators do. Love him or hate him, he's bringing something new to the table. And it threatens what we might call the "Morning Joe Model," in which smart people share their opinions based on their experience and their gut. Silver threatens to put them out of a job. No wonder they aren't entirely rational.
On the morning of election day, CNN was saying the race for the White House "is in a dead heat," while Silver was reporting that Obama had a 90.9 percent chance of winning. Obama's win does not mean that Silver was right and CNN was wrong; we're dealing in uncertainty in all of these predictions. But if Engber argues that the pollsters got it right, not Silver, then what polls does he think CNN was reading?
At Forbes, John McQuaid, a former reporter for The Times-Picayune of New Orleans, spells out the issues nicely in a post entitled "Three Lessons for the Nate Silver Controversy." First, he writes, "the modelers are here to stay." He's right. Next, he notes that "elections are less surprising than most of us think." This echoes Engber on the polls, but McQuaid doesn't share Engber's indignation. And third, he agrees with me about why the traditional analysts find Silver threatening:
The silly debate over Silver’s model showed a political, pundit and media class at a classic horse-and-buggy meets internal combustion engine (or is it ink-stained newspaper curmudgeons meet Internet?) moment: ignorance and fear masked by loud harumphing. It seems like an amusing footnote now, but only days ago Peggy Noonan was seriously arguing that a mild proliferation of Romney yard signs she observed in liberal Northwest Washington, plus magical vibrations of true American-ness emanating from all over, were signs of imminent Romney victory.
You can read scores of analyses of Silver's predictions on the web, but they mostly cluster around the issues I've mentioned here. And we await Silver's further analysis of the election. At 2:31 a.m. on the night of the election, he posted his final item before "going to get some sleep and grab a beer." Presumably not in that order.
It's now the day after the election. Wake up, Silver; we want more numbers. And we want them now.
This post originally appeared at the Knight Science Journalism Tracker.