Of the many stories in Democratic pollster Stan Greenberg's new book, Dispatches from the War Room, the most newsworthy may be his slashing condemnation of Mark Penn, the pollster that displaced him within the inner circles of both President Bill Clinton and British Prime Minister Tony Blair.
Greenberg, reacting to what he saw of Penn's polling when both worked on Blair's 2005 re-election, describes Penn's methods as "errant," his tests of messages as "biased" and "rigged" and his documentation lacking "transparency" and "the information normally delivered by a professional research organization."
Penn, in response to my query, defends his surveys for the Labor Party as "extremely accurate," says Greenberg was excluded from information because he was "not in the loop," and describes Greenberg's attacks as inaccurate and "unsubstantiated."
This exchange brings out into the open a particular critique of Penn that until now came mostly from speculation or the reports of anonymous sources. Some aspects of their stories are in conflict, if nothing else the undisputed facts illustrate the contrast between the Carville/Greenberg "War Room" model of campaigns and the style of consulting that Penn practices.
[This post is part of Pollster.com's series on Dispatches from the War Room].
* * * *
For those just joining this conversation, Greenberg's new book chronicles his work for five national leaders, including Clinton and Blair. The story starts with the 1992 campaign that put Clinton in the White House and made the "War Room" famous. After the 1994 elections that swept Republicans into control of both the House and Senate, however, Clinton started taking advice from Dick Morris and brought in pollster Mark Penn to replace Greenberg and conduct polling for the 1996 reelection campaign.
In the midst of the first Clinton term, Greenberg also went to work for Britain's Labour Party, helping to create another political "War Room" that helped Blair and Labour win landslide victories in 1997 and 2001. Following the terrorist attacks of September 11, 2001, Blair emerged as one of the closest allies of the U.S. and President George Bush in both the response to the attacks and the Iraq War. As Greenberg relates in the book, he personally opposed the Iraq war and felt uncomfortable with Blair's embrace of Bush and military action in Iraq. Nevertheless, he remained an advisor since he "did not question Blair's motivation" nor his commitment to the larger "political project" that "transcended this war."
Ultimately, however, the war took a toll on both Blair's popularity and his relationship with Greenberg. In this period, Greenberg recounts several examples of Blair rejecting his counsel as he struggled with how to win back public support and whether to seek reelection. In November 2004, after Blair decided to seek a third term, the Labour Party informed Greenberg that they had retained Mark Penn and that Blair wanted "the polling program split between the two of you."
Greenberg soon learned that during periods of Blair's doubts, "President Clinton and Hillary, too, called repeatedly to urge Blair to run again...They also pushed for Blair to use Mark Penn to help find a way out." Penn had conducted polls kept secret from Blair's inner circle, even from Phillip Gould, the polling and strategy advisor. Friends also told him that the plan had been to "fire" him, yet "somehow nobody did it." So he continued to conduct polls on behalf of Labour in marginal districts and, as such, remained present in the campaign,
watching it all from a dysfunctional new war room. The Penn people were grouped at one set of long tables and my people at another, a touch further away from the cluster of tables at the center. It was a daily humiliating experience, but I decided I had too much invested in "New Labour" to walk away.
Greenberg's criticism of Penn stems from what he observed during the period including, by his account, both the shift in message and the results of Penn's polling. He recounts an episode in March 2005 in which Penn's polls showed Labour "with landslide leads," while Greenberg's surveys in marginal districts were closer and his review of public polls showed Labour leading by only four percentage points. A memo from Greenberg to Blair about these numbers "led Philip [Gould] to erupt in the war room," shouting at Greenberg's staffer to "tell Stan these emails to Tony are wrong and very destructive." In the end, Labour won by just three percentage points and a significantly reduced majority.
After the election Greenberg wrote three memos to Blair and then Chancellor Gordon Brown that interpreted the results and "put on paper in an inescapable way my outrage at the polling and the banal strategy that followed from it that put Blair and Labour at risk." He describes the overall "research program" (which included work by Penn and by others) as "unprofessional and lacking in methodological rigor...biased, self-deluding and overly optimistic."
The memoranda, as recounted in the book, include these specific references to Penn's work:
Penn's errant polling methods had produced landslide predictions all along, giving way in the last week to a roller coaster: first Labour crashed, then it surged back. One could only divine the cause because Penn's firm provided none of the information normally delivered by a professional research organization," I wrote. The lack of transparency allowed findings to be "packaged in scientific surety that should never have been reported with anything but the greatest qualifications."
The biggest problem was that the campaign fixated on winning over some Americanized target group. Without any evidence, Penn touted that "soccer moms" were the key swing group in the 2004 U.S. elections. In Britain that group became "married mums." The problem was that three quarters of the "lapsed Labour voters" in the marginal seats did not have kids at home. Penn set the campaign on a course that missed most of its key targets and would not adapt. So, "during the campaign, we successfully raised Labour's support with women with children at home" but "that came at a price" among older voters and pensioners and men under forty-five who were not stirred by yet one more initiative on breast cancer screening and childhood obesity. "It was as if creating an American-style gender gap was something to emulate, when Democrats in the U.S. have had so much difficulty winning nationally." I concluded that this mindless, fixed theory had "consequences": "a reduced Labour vote share and a reduced parliamentary majority."
After some criticism of focus groups by the campaign (but not by Penn's firm), Greenberg concludes, again quoting from his own memorandum:
[T]o keep the campaign on its predetermined course, "the national survey's were riddled with questions whose biased wording seemed to get the reports to a preferred conclusion." In short, the tests were rigged.
What makes these accusations unique is that they are from an on-the-record, first person account. Similar accusations -- such as not not sharing "filled in" questionnaires (also known as "marginals") and crosstabs or cherry picking results to support a favored position -- have bubbled up in the past, but Greenberg is the first I can recall willing to make such accusations out in the open.
I emailed Mark Penn for comment and he sent a response that characterizes Greenberg's charges "unfortunate" and "unsubstantiated," defends his firms' data and its contribution to Labour's 2005 campaign. "We did in fact change the course of the campaign, developed a new message, a new set of targets, were extremely accurate, and received extensive written and personal praise from the prime minister." He denies as "ludicrous" the accusation that his polls showed Labour winning by landslide margins: "We never predicted a landslide. Most of our polls showed Labour and Tories within 4-5 pts. It was Stan's predictions of a loss that proved inaccurate."
Penn's complete response appears after the jump, but the following paragraphs are most relevant:
First and perhaps most disturbing is the idea that he did not receive any standard market research information - of course HE didn't. The campaign received all of the agendas, marginals and crosstabs as requested without reservation. But Stan didn't because he was not in the loop as 1) many of the polls contained highly sensitive questions and 2) he was seen as highly adversarial looking only to undermine the team and their effort - which is rather borne out by his book. The memo he quotes in his book actually sealed his fate and was seen as highly unprofessional and mocking of an effort led by a hard-working team of smart professionals including Philip and Alistair Campbell. Blair specifically praised the professionalism, quality and creativity of the work.
Penn may have had a different memo in mind, but to be clear, the internal memos that Greenberg quotes in the passages above were written two weeks after the 2005 elections. Penn continues:
Again, Stan was excluded from the strategy sessions and all meetings with the prime minister and provided with only limited information at the specific instructions of the client. How could he have understood what was going on when wasn't there for the high-level discussions? This is rather well illustrated by his saying he learned of us in December and that Philip Gould did not know of our work. In fact, we started many months earlier in the summer when the most in-depth positioning and clustering work was done and Philip was at every presentation and supervised us. Of course he did not tell Stan that he was working with us long before or that he had all of the information on the polls for reasons that are quite obvious.
And on top of all this Stan has confused daily tracking in the last month that was done by another firm (not Penn and Schoen) that Philip had arranged to have shared with the campaign. This daily polling did tend to shift around and we did not have much information on it - it was not used for much of anything. We did not do daily tracking - we did the message and strategy polling, and we supplied the questions, the answers, the sample sizes, and the logic behind each and every conclusion or suggestion.
We worked almost a year on this campaign asking hundreds of questions of all types and nothing was "rigged." Philip had input to all of the questionnaires without reservation as did members of the team and he ran extensive independent almost daily focus groups alongside us.
There are clearly some facts in dispute in this exchange, and resolving the conflicting stories is beyond the scope of this post. Other issues, such as whether question wording is slanted or "rigged," will inevitably involve subjective judgments. I assume that he will have more to say in response, and we look forward to trying to dig deeper into this controversy.
Timing is important in all of this. If "the campaign received all of the agendas, marginals and crosstabs as requested without reservation," when exactly were they requested and received? The complaint I heard from within Hillary Clinton's campaign was that Penn only shared filled-in questionnaires with the campaign's senior staff after the meetings or conference calls at which results were presented and decisions made, and only shared cross-tabs reluctantly on the condition they be locked up in the campaign manager's office.
But if we set aside the conflicting aspects of the two pollsters' accounts, we can still see a clear contrast in the way they approach campaigns and a vivid illustration of the Carville/Greenberg "War Room" philosophy. As Greenberg writes, James Carville's vision was to "house...all the key campaign people [in a single room], the opposition researchers, policy people, and news monitors." Yes, as most of us know, all of it was "geared toward fast analysis and instant response," but speed alone was not what made the concept unique.
What really distinguished a Carville War Room was the absence of doors, the way information flowed freely among the staff that worked inside. Instead of limiting access to "sensitive" polling information to the half-dozen or so in the campaign's "inner circle," Carville opened access to everyone working in the War Room, including relatively junior staff. "The idea was that information was widely shared among the campaign people," Carville himself told me when when I spoke to him on Friday. Even though he routinely shared sensitive strategic information with hundred or so staffers in the Clinton campaign "we never had leaks" (though to be clear: Carville was not involved in the 2005 Labour campaign).
In Penn's conception, polling information should be shared only with a select few with the candidate's ear, those "in the loop." Carville and Greenberg's "war room thinking" trusts everyone in that room with the details of strategy and polling so they can be, as Greenberg puts it, "sensitive to all kinds of information, and anxious to jump on any sign of strategy going awry."
Trust your staff with the data that drives the strategy and they are more likely to understand it and react accordingly. Limit "the loop" to just a small handful in a presidential campaign and expect infighting, dysfunction and poor execution to follow.
[Past relationship disclosed: I worked in Stan Greenberg's company, then known as Greenberg-Lake: The Analysis Group, in 1990 and 1991].
Update: Greenberg responds to Penn, Penn responds to Greenberg.
Greenberg's full passage from Dispatches on this subject and Penn's response follow after the jump.
Stan Greenberg's account from Dispatches From the War Room, pp 265-266:
Two
weeks after the election, I wrote three memos to Blair and Brown to
explain the voting patterns and dynamics in the election and their
implication for future strategy. I knew, of course, that I would not be
part of that strategy unless Brown succeeded to Number 10, 1 wrote the
memos partly for closure, but more to put on paper in an inescapable
way my outrage at the polling and the banal strategy that followed from
it that put Blair and Labour at risk. Jeremy and the whole Labour team
joined in the cathartic analysis that we felt morally compelled to
write.I thought carefully about my characterization but decided not to
qualify it: "The research program was unprofessional and lacking in
methodological rigor, though it asserted a scientific certainty; it was
erratic but rigid at the same time; it was biased, self-deluding and
overly optimistic; it lacked both transparency and accountability."
Even as I was writing I was thinking, as I had during the campaign, how
can such quality people have any patience with this kind of work?Penn's errant polling methods had produced landslide predictions
all along, giving way in the last week to a roller coaster: first
Labour crashed, then it surged back. One could only divine the cause
because Penn's firm provided none of the information normally delivered
by a professional research organization," I wrote. The lack of
transparency allowed findings to be "packaged in scientific surety that
should never have been reported with anything but the greatest
qualifications."The biggest problem was that the campaign fixated on winning over
some Americanized target group. Without any evidence, Penn touted that
"soccer moms" were the key swing group in the 2004 U.S. elections. In
Britain that group became "married mums." The problem was that three
quarters of the "lapsed Labour voters" in the marginal seats did not
have kids at home. Penn set the campaign on a course that missed most
of its key targets and would not adapt. So, "during the campaign, we
successfully raised Labour's support with women with children at home"
but "that came at a price" among older voters and pensioners and men
under forty-five who were not stirred by yet one more initiative on
breast cancer screening and childhood obesity. "It was as if creating
an American-style gender gap was something to emulate, when Democrats
in the U.S. have had so much difficulty winning nationally." I
concluded that this mindless, fixed theory had "consequences": "a
reduced Labour vote share and a reduced parliamentary majority.""It had a theory of the race and only sought information that
allowed it to elaborate its plan or deepen support with its predefined
target audiences," I wrote. Accordingly, "it only conducted focus
groups with mums and dads, and when groups were finally held with older
voters, they were so hostile the campaign decided not to do them
again." And to keep the campaign on its predetermined course, "the
national survey's were riddled with questions whose biased wording
seemed to get the reports to a preferred conclusion." In short, the
tests were rigged.The rigidity meant that Labour lost its ability to do what I do:
listen to people and learn. The research program was totally at odds
with "war room thinking." "The successful and winning campaigns we've
experienced are focused, disciplined, sensitive to all kinds of
information, and anxious to jump on any sign of a strategy going awry
or not proving effective enough," I wrote. In a campaign, bad news
"concentrates the mind.""The result was a surprisingly rigid and conservative campaign,
unwilling to risk expanding on the economic narrative or taking the
gamble of really offering people a hopeful vision for the future.
Mark Penn's response, via email:
Stan Greenberg has done some very good work at critical times in history and he
need not try to diminish the good work of others to raise his own. We have all
won and all had losses. We all have had clients use us and clients decide to
use others. It is unfortunate he made these unsubstantiated attacks from
what must have been a very frustrating vantage point.The facts are simple and straightforward - PSB was brought in at a low
point when Blair believed the race was likely lost and he decided a new team,
new path and message were needed to turn it around. We replaced Stan as
the pollster for the prime minister while he continued to do limited work for
Brown and the party. We did in fact change the course of the campaign,
developed a new message, a new set of targets, were extremely accurate, and
received extensive written and personal praise from the prime minister. We
continued to work after the election for the pm while Stan's arrangement
with the party was terminated.When Blair left and
Brown came in, Stan had an unbridled chance to test his theories on his own
- and he was subsequently let go by Brown and the Party. Brown
now finds himself 20 points behind.First and perhaps most disturbing is the idea that he did not receive any
standard market research information - of course HE didn't. The
campaign received all of the agendas, marginals and crosstabs as requested
without reservation. But Stan didn't because he was not in the loop as 1)
many of the polls contained highly sensitive questions and 2) he was seen as
highly adversarial looking only to undermine the team and their effort -
which is rather borne out by his book. The memo he quotes in his book actually
sealed his fate and was seen as highly unprofessional and mocking of an effort
led by a hard-working team of smart professionals including
Philip and Alistair Campbell. Blair specifically praised the
professionalism, quality and creativity of the work.Again,
Stan was excluded from the strategy sessions and all meetings with the
prime minister and provided with only limited information at the specific
instructions of the client. How could he have understood what was going on when
wasn't there for the high-level discussions? This is rather well
illustrated by his saying he learned of us in December and that Philip
Gould did not know of our work. In fact, we started many months earlier
in the summer when the most in-depth positioning and clustering work was done and
Philip was at every presentation and supervised us. Of course he did not
tell Stan that he was working with us long before or that he had all of the
information on the polls for reasons that are quite obvious.And on top of all this Stan has confused daily tracking in the last month that
was done by another firm (not Penn and Schoen)
that Philip had arranged to have shared with the campaign. This daily
polling did tend to shift around and we did not have much information on it
- it was not used for much of anything. We did not do daily
tracking - we did the message and strategy polling, and we supplied the
questions, the answers, the sample sizes, and the logic behind each and every
conclusion or suggestion.We worked almost a year on this campaign asking hundreds of questions of all
types and nothing was "rigged." Philip had input to all of
the questionnaires without reservation as did members of the team and he ran extensive independent almost daily focus
groups alongside us. We pioneered also companion internet diary polling
as well. The target clusters were based on rigorous statistical analysis
and mapping with the help of a Princeton PHD in statistics and lecturer.The third term win was a historic first for Labour even as 70% opposed Blair's Iraq policy. We focused the campaign message on the successful economic work of the administration for the people of the country and contrasting what the future would be like under the Conservatives vs. Labour. The campaign theme - "forward not back" - called on Britons to continue the progress made by Tony in the economy, healthcare, government and in so many other areas.
In the final 10 days there was a significant "October surprise" of confidential, classified Downing Street memos being leaked to the press that showed Blair's early and unwavering support for Bush on Iraq that brought us down a few points but the campaign responded and held the line to win.
Here are some of the specific inaccuracies:
P 262 - Philip Gould did not know about our work. Fact: He supervised it and was at every presentation.
- It was a "banal strategy that put Labour at risk" Fact: it was a winning strategy that by focusing on the economy and the future turned Labour's position around and helped to deliver Labour's historic third term victory
- "lacking in methodological rigor" - In fact, sophisticated clustering methods were deployed; The polling accurately predicted both the key audiences to win and the likely election result. On the final day we all put our predictions on a campaign chalk board; Stan was off; PSB was accurate.
- "landslide predictions all along" - Ludicrous. We never predicted a landslide. Most of our polls showed Labour and Tories within 4-5 pts. It was stan's predictions of a loss that proved inaccurate. We were not responsible for the daily tracking he seems to talk about.
- "Penn followed an American model - soccer moms" Fact: the campaign used PSB polls to identify key target groups - one of which was 'school gate mums'. This was one of a number of target groups, including pensioners, younger voters, lapsed labour voters, and weak Liberal Party voters. But the question was not only about which voters had been lost in the years before, but which voters Blair could now pickup against a very cold and austere opposition candidate like Howard. Women with kids did prove an important target and it was well substantiated in the polling, cluster analysis and the voting gain - unsubstantiated were Stan's theories. He had done all the polling for years while older voters and women both shifted against Blair and we had to pick up the hand where we found it and recover in the shortest possible time.
- the campaign only did groups with mums and dads - Fact: the campaign held groups with all manner of voters and demographics - Lapsed labour, LD voters, men, women, mums, dads, young people. They were done almost daily and run by Philip Gould.
- "unwilling to risk expanding on the economic narrative or taking the gamble of really offering people a hopeful vision for the future" Fact: The frame of the campaign was 'forward not back', a line devised by me and speaking entirely to the need to focus on the future. That Blair had modernized the country, the government and the economy - and we had to keep on that path -- was the central thrust of the campaign.