This blog post (originally published in April 2015) is the first of two parts about alternative metrics.
“What you measure affects what you do. If you don’t measure the right thing, you don’t do the right thing.”
— Nobel Prize Winning Economist, Joseph Stiglitz
Part 1 — The Shortcomings of Traditional Metrics
When gauging the value and success of any business, product, or service, people usually talk about metrics such as growth in revenue, profit, stock price, or market share. These hard quantitative metrics are great tools for measuring some things, but a strict reliance on them is akin to judging a professional athlete’s abilities based solely on their caloric intake and resting heart rate.
I’ve never seen a company’s mission statement mention share price. However, like treating the symptoms instead of the disease, a fixation on financial metrics can supplant, and may even undermine the goals a company is supposedly working toward. These unintended consequences (technically known as “perverse incentives”1) can be avoided by developing and employing alternative metrics that allow you to clearly represent and understand:
- Your goals
- The steps required to reach your goals
- The measurable indicators that correlate to your progress and success in each step
- The warning signs if things start heading off-track
The importance and nuance of alternative metrics can be demonstrated by comparing business performances to those of professional athletes: Every cyclist competing in the Tour de France may appear to be a gleaming example of medical health and fitness. However, (somewhat amusingly) neither their current health nor their past cycling performance is in any way indicative of their golfing abilities. Even when competing in the correct sport, at the end of the day, some cyclists will be deemed winners, while others fade into obscurity- or worse, infamy. It’s not easy to predict these outcomes before a race, and as seen in the case of Lance Armstrong2, the apparent reasons and initial assumptions for high quality performance may be deceptive. Similarly, it’s not easy for businesses to discern which metrics, strengths, or prior successes are the most strongly correlated to their future success. Furthermore, even assuming they’ve made the right choices, targets can be artificially or unnaturally reached through unethical means.
The current health and past cycling performance of Tour de France competitors is in no way indicative of their golfing abilities
Alternatively, consider NFL and NBA players — although their high performance is handsomely rewarded at the time, their correspondingly lavish lifestyles and physically taxing careers leave most players with futures fraught with severe health and financial issues.3 Similarly, unsustainable or aggressive short-term business goals may conflict with or undermine a business’s long-term goals.
- If you’re only tracking hard quantitative business outputs and metrics, you’re not seeing the whole picture.
- Regardless of the reason, whenever metrics become unmoored from the goals and successes they’re supposed to be indicative of, they cease to be helpful.
In part 2, ‘Experiences aren’t Quantities’ we’ll discuss various measuring tools and different kinds of value.
Wikipedia has a short but entertaining list of examples from history. Highly recommended.
A cancer survivor who set a record by winning the Tour de France seven times in a row, but was later stripped of his titles and banned from all future competitions after being charged with having illegally used performance-enhancing drugs.
NFL players often retire in their early thirties, and ~80% of them are completely broke three years later. (Source)