As a rule, organizations work toward the goals they set. When these goals have numbers attached to them, the work focuses on what drives these numbers to improve. Unfortunately if these numbers are based on vanity metrics, an organization is bound to get sick. Vanity metrics, a term used by The Lean Startup author Eric Reis, are metrics that only look at the surface performance and don't relate to the actual health of an organization. The same is true for a person, as I learned the hard way.
I was recently diagnosed with an illness that came as a direct result of stress and lifestyle. After looking back and examining the "data," I now realize that all of the deep goals and measures I had set for myself were work related. All of them.
According to my vanity metrics, my graph was making a nice "up and to the right" trend. I looked healthy, ate well and worked out every week. But by ignoring the deeper measures that contribute to a healthy system, I was unknowingly wearing my engine down. (I have since added fitness, relationship, and vacation goals into my measures of personal success).
You're so vain, you probably think this metric is about you
The easiest way to understand a vanity metric is to remember that "metrics are people too." Every chart, trend or spreadsheet should be presented in the appropriate context to give us an aggregate understanding of how people are actually behaving. This means that statistics like number of page views, users, or Facebook fans aren't, on their own, telling you what people are doing and don't necessarily indicate the health of an organization.
For example, on an organization's Facebook fan page, total Likes is the easiest thing to measure and grow. But when we think of this metric from a "people perspective," it is clear that what we should care about and measure is not simply the number of Likes, but the people talking about the fan page (demonstrated through a combination of active Likes of posts, shares and comments). Consider two fan pages. One has 3.3 million likes and 3,300 people talking about it (Chase Community Giving). The other has 200,000 Likes and 5,000 talking about it (DoSomething.org). Which is healthier? Based on recent Facebook changes associated with the "timeline" format, the answer is the fan page with fewer total likes, but more active users, because moving forward a user that isn't talking about a page will rarely see the activity feed from it.
The same holds true for the number of page views or "hits" for a website. Though this is one of the first numbers Google Analytics serves up in a dashboard, it is the first one an organization should ignore. An organization that bases it's goal on total traffic is comparable to a store that only cares about how many times people walk in or how many things they look at when they do. A store that did this would be out of business very quickly because they aren't tracking whether or not people are actually buying things. Metrics like bounce rate (percent of people that leave immediately from a page), time on site, goal conversions and social shares of content come much closer to demonstrating what people actually like. Avinash Kaushik does a great job explaining how web analytics should focus on outcomes.
Total users is another classic vanity metric that can give a quick idea of the size of an organization but no sense of actual health. Like trying to drive a car by just looking at the odometer, this metric gives no insight into the rate that users are increasing, decreasing or becoming inactive. In the start-up world, the total users metric is a joke partially due to books like The Lean Startup and the fact that venture firms know that this metric doesn't put money in the bank. Silicon Valley start-ups live and die by acquisition rate, active users, referral rate (number of people referred per user), and churn (users leaving). Not-for-profits with online communities should be no different, even if the funders don't care about it.
In the same way that I made myself sick by focusing on a narrow band of vanity metrics, so too can an organization work itself into an illness. However, the solution is simple: put the people back in metrics and pay attention to the true health of a system if you want to make change. Vacations don't hurt either.