THE BLOG
11/20/2014 04:31 pm ET Updated Dec 06, 2017

How What We Measure Doesn't Measure Up

2014-11-20-measuringtape.jpg

As if it weren't bad enough that the Information Age deluged us with data, now we're finding out that some of our favorite metrics aren't telling us what we really want to know.

Take economic prosperity, for example. Gross Domestic Product (GDP) is generally the go-to fact when we want to gauge a country's prosperity. But does the "monetary value of all goods and services produced on an annual basis" really capture how well we're doing on those most-important things that money can't buy? What about freedom, family, democracy, jobs, opportunity, health, education, safety? GDP captures none of these.

That's the rationale behind the Prosperity Index, a project of the Legatum Institute, which recently issued its sixth annual report. Legatum makes the case that measuring prosperity requires a multi-dimensional approach, and they have built an index on eight core pillars: economy, entrepreneurship and opportunity, governance, education, health, safety and security, personal freedom, and social capital. They rated and ranked 142 countries on these measures.

At the top of the 2014 prosperity scale are some likely suspects: Norway, Switzerland, and New Zealand. No surprises at the bottom, either: Central African Republic, Chad, and the Democratic Republic of the Congo.

But while those faring best and worst tend to be consistent across all subcategories, there are some important distinctions that illustrate how a single measure of prosperity just doesn't measure up. China, for example, ranks 6th in economy, but 117th in personal freedom. Uzbekistan is 16th in social capital, and 118th in governance. Israel is 28th in entrepreneurship and opportunity, but 105th in safety and security.

The U.S., UK, and Germany stayed generally steady in 10th, 13th, and 16th positions in the six years of the Prosperity Index reports (2009-2014). But other countries experienced remarkable progress or decline. Rwanda, Indonesia, and Sri Lanka each moved several places up on the international rankings, for example. Greece, India and Syria dropped significantly.

Brookings convened a panel earlier this week to discuss the 2014 Prosperity Index report, "Defining Opportunity and Prosperity Beyond Economic Growth." Five international authorities reflected on the data and launched interesting discussions around the kind of questions that naturally emerge when you've got nuanced information: Is it possible to have high levels of income inequality and quality of life at the same time? Why do some countries fare poorly on the objective indicators, but well on the subjective ones? Does it make a difference if public goods are provided by government or the private sector?

But one of the most interesting observations was very basic. A couple of panelists suggested that the nine top-ranking countries on the index are all relatively "boring" places, less dynamic and vibrant than several others that rank much lower overall. Is it possible that even a complex index like Prosperity can't capture everything?

This brought to mind another interesting ranking of global places that came out this past week and likely caused a number of metrics enthusiasts to scratch their heads: the GOOD City Index. This second annual listing is put together by GOOD magazine, which believes "that a city's heartbeat is best measured in 'possibility' -- the pervading sense that though a place may be far from perfect, its citizens are taking a bold stake in its future through a mixture of creativity, hustle, and civic engagement."

GOOD identified eight measures of "possibility" that include a city's ability to flourish during a crisis, and its levels of civic engagement, street life, connectivity, diversity, work / life balance. The highest ranking city this year was Hong Kong, followed by Johannesburg and Mexico City. Only two American cities made it on the top-20 list: New Orleans and Detroit.

Go figure? Yes, I believe so.

Here's one takeaway. While uncertainty is not what we want among structural engineers and brain surgeons, might a little ambiguity actually be healthy for the rest of us? Might we be better off if we weren't so quick to be sure of ourselves, so eager to beat others up with our facts, so righteous in our certainty about who's better than whom? I'm not sure, but it wouldn't hurt to think about it.