Who can argue with the evaluations expert who challenged me over coffee, "Management needs data for informed decision-making." The smiling, but incredulous, tone implied that the opposite position (mine, I guess) is willful, ignorant, unruly.
In the fierce world of economic injustice and grinding poverty, the rationale for social impact metrics and rigorous program evaluation is logically irrefutable -- and pragmatically beside the point. Here are few practical considerations and concerns:
In the real world, decision-making is based on imperfect information. Decisions are time dated. Complete information is preferable, but tomorrow relentlessly produces countervailing facts and perspectives, not to mention Monday morning quarterbacks (now called external evaluators). Whatever the ideal decision-making data set, indecision is itself a decision with consequences.
In the face of economic and social injustice, Martin Luther King spoke about "the fierce urgency of now." He did not speak about the fierce urgency of developing social impact metrics or evaluation indices to determine if civil disobedience is cost-effective relative to other civil rights interventions. If he had, "I have a dream" would have been "I have an evaluation report."
There are limits to how much a social enterprise leader can do. Capital fundraising, vision articulation, human resources, inspirational leader, board management, stakeholder engagement, program management, product development, financial reporting, media affairs, legal matters, etc. are typical responsibilities of the average social enterprise CEO. Evaluation requires a CEO's most precious commodity, time. Evaluators demand management attention to frame the project, review interim updates and study the findings (and, of course, listen to the next consulting contract pitch).
Moreover, evaluators need rigorous evaluation. Of the numerous evaluation methodologies, which is most appropriate in any given situation? And, once that is determined, which consulting firm or university institute is the most accomplished at it?
And, let's not be naïve. Will the evaluation group's reputation sufficiently impress donors and other external audiences demanding "proof of results"? Will the study at least insulate the organization from criticism, a defensive cover-your-ass response to the evaluation craze? Will the selected evaluator "cooperate" to assure the findings are not punitive, harshly written or otherwise jeopardize future funding?
Speaking of a limited resource, Huffington Post blogger Sashar replies to my earlier blog,
I want to know who is supposed to fund these social impact studies? I am a small international funder who makes site visits to all my grantees... Site visits give me a firsthand view of the work being done on the ground. My evaluations are informed by financials and due diligence before and during my visit, but at least 50 percent is based on intuition and gut instinct.
As Albert Einstein, a fairly distinguished scientist himself, noted, "The only real valuable thing is intuition."
Whatever the utopian plans of foundations, government funders and celebrity academics, external social impact evaluations & audits -- like the annual Kabuki theatre of external financial audits -- are in all probability going to remain an expensive, sporadic management distraction.