Ezra Klein really hates the new book by Clinton pollster Mark
Penn:
[Penn's] new book Microtrends
is so bad that the question--in a fair world--isn't whether it will destroy his
own reputation, but whether it is so epically awful as to take the entire
polling industry down with it.
That certainly grabbed my attention. Klein expands on that
idea in his last few paragraphs:
Pollsters occupy a uniquely powerful space in American
political discourse: They bring science to elections. Armed with heaps of raw
data, they elevate their opinions into something altogether weightier:
Conclusions. When an organization sends out a press release saying the
organization is right, it's ignored. When a pollster sends out a poll showing
the electorate agrees, ears in Washington
perk up.The enterprise has always been dodgy. Populist
pollsters reliably discover that the electorate thirsts for more populism.
Conservative pollsters routinely discover a small government consensus pulsing
at the heart of the body politic. When the libertarian Cato Institute
commissioned a poll of the electorate, they found--shockingly--that the
essential swing vote was made of libertarians. Remarkably, whenever a politician
or self-interested institution releases a poll, the results show a symmetry
between the attitudes of the pollster's employer and those of the voters. But
Penn's book shines light on this phenomenon: If he is the pinnacle of his
profession, then the profession uses numbers as a ruse--a superficial
empiricism that obscures garden-variety hackery. And that's a trend worth
worrying about.
I have not yet read Penn's book, and like Klein, have never
met him. But my own sense is that Penn is an unusual case as political
pollsters go, both in terms of his paycheck and methods
(for example, as Klein notes, "unlike most pollsters, Penn never releases his
raw numbers, only his analysis").
But set Penn aside for a moment. Klein raises a fair point
about the many publicly released surveys sponsored by partisans and interest
groups. Consumers should always approach such data with skepticism because --
surprise, surprise -- interest groups and their pollsters tend to cherry pick results
that make the most compelling case for their side. Educated consumers confronted
with such releases should always wonder, "what results am I not seeing?"
Still, I think Klein goes a bit too far here. His conditional rhetoric --
"If [Penn] is the pinnacle of his profession, then the profession uses numbers
as a ruse" -- strikes me as the same sort of speculative leap (based on a sample
size of n=1) that Klein finds so troubling in Penn's book.
I am a pollster, of course, so some bias on this issue is
inevitable. Readers, what do you think? Is Klein's criticism (of Penn and pollsters generally) fair? Is Penn "the pinnacle" of the polling
profession?
Update - Mark Penn emails:
Given all the reviews of Microtrends I am rather
surprised the only one you mention is Ezra Klein.Would appreciate your mentioning or linking to some
of the reviews in USA
Today, Business
Week, Economist,
Bloomberg,
Politco, Newsweek among others
that had a very different and very high opinion of the book.It is unfortunate but not surprising that an
American Prospect writer put out a review like this given their past articles. Of
course that would be a correlation, not necessarily causation. Or is it?It is not at all a political book and I hope you
will read it and enjoy it.
Fair enough. I added the links above. For what it's worth, I did link
to the Politico review a week ago.