A few boys are laughing with me as we talk cricket, leaning on a broken handpump. Of course, they love their Indian national team and weren't having my nonsense about how South Africa is a better team.
The handpump is important to them. Not because water flows from it as hoped -- actually it has been broken for years. The boys said it had only worked a short time. The handpump is now one of their wickets (essential to cricket, you don't want the ball to hit the wicket). They like it because it pings whenever a ball hits it, reducing the conflict that inevitably arises in sport when players debate whether a batter is out or not. The handpump brings great joy to the boys, who generally do not fetch water -- "my sister is responsible for water" is a common refrain -- but the boys can always use sports equipment.
An encased plaque at the base of the handpump shows exactly who provided them with their wicket, proudly listing donors and implementing agencies that were bringing water to this village. The plaque is now a slight nuisance -- better to just have the handpump wicket as the plaque because it sometimes trips up the wicket keeper as he tries to make a play.
Joe Saxton recently wrote a blog about the new, unnamed "impact thought police" that inappropriately pushes charities to measure outcomes. He suggests that the emphasis on results undermines charities, smaller charities in particular. His argument is common in the non-profit sector:
"The problem is that outputs are easy to measure whereas outcomes are hellishly difficult. I'd rather a charity wore its outputs on its sleeve than did nothing ... I'd like a charity to brag about its outputs: how many meals it has served to homeless people, or how many people have called its helpline ... If we wait for them to brag about their outcomes we will all be grey and old."
The handpump wicket is a perfect example of the flaws in this argument. In fact, a large charity did exactly what Saxton suggests -- they reported on the installation of this handpump in rural India, took photos and testimonials of "beneficiaries," packaged that with clean financial statements and (I am sure) a request for further funding based on their success at implementing a "sustainable water project."
The focus on outputs over outcomes is precisely why the international charitable water sector "market" is so distorted -- where good stories, celebrity endorsements and slick PR take precedent over quality work that is monitored over time. Short-term outputs, like numbers of beneficiaries supported in a given year, number of water points or latrines constructed, and percentage of loans repaid all focus on short-term results.
Connect output-focused indicators like these with the wreckage of water and sanitation infrastructure in Africa, Asia and Latin America, as I have discussed here, and now we have a sector that is not solving problems but banging in infrastructure.
The concept that outcomes take too long to measure is a common and poor argument. In water supply there are various indicators that are clear and measurable, which suggest whether a system is on track to being viable and sustainable over time. These indicators are not hard to collect regardless of organizational size, and do suggest whether the system installed is likely to last and be impactful.
- Issues of finance and management -- money in an account; enough money to repair and eventually replace the water system;
- Functionality -- is water flowing; how long do repairs take;
- Access -- who collects from the water point; who does not and why; and
- Quality of service -- water quality, water quantity and availability.
By tracking these indicators over time, we shift from looking at short-term results to insights that shed critical light on long-term outcomes.
Finally, output-focused aid is the complete opposite of what most charities actually profess to support. No water sector agency pitches its work to potential donors on outputs -- "finance us and we will install 100 water points in rural India and hope they last." Charities market their work on outcomes. Buzzwords like transformative, life-saving, and sustainable are cornerstones of charity messaging and fundraising. As such, charities should be assessed on whether those messages prove to be true, and when not accurate, on what programmatic changes will be tested to change course and hopefully to achieve those ambitious results.
It's not a cost issue as Saxton implies. It's a focus issue. Good agencies that track their work, and are willing to learn by focusing on outcomes generally improve and innovate. Those that do not, and stick behind outputs as their marker of success generally stagnate or get called out eventually.
All you have to do is read the plaque to see what I suggest is true, if the boys will stop playing long enough for you to get a good look.
I welcome your comments, please join the conversation @NedBreslin, and learn more about Water For People's Everyone Forever approach.