The Ultimate SaaS Metric is No Friend of Reality
3 MIN READ
|
Updated on December 19, 2017
In countless minds, the search for the ultimate SaaS metric continues apace. Mostly, to no effect. Why, though? Isn’t there a be-all-end-all metric? Can’t there be a proxy for everything?
An answer resides, to a great degree, in understanding something that this elusive metric will always have to grapple with: Reality.
The natural aim of metrics (and other proxies) – obedient to our expectations – is to mask the complex medium of reality, by serving us a simple model of the way the world works.
It helps.
Measurement replaces intuition, which is often fallible. It replaces trust, which is often misplaced. It finesses complexity, which is frequently irreducible. So faulty intuition, untrusted partners, and complex systems can be understood via intuitive, trustworthy, simple metrics.
Though, despite the benefits that the needles of our precision instruments point at, David argues, the act of measuring hides a good deal from us, as it “obscures, disrupts, and distorts systems.”
When reality recollects the things that annoy it most, our over-reliance on proxies is placed at the very beginning.
Consider the case of Population Density.
Alaska’s score is 1. That means, you’ll find one person for each square mile of land. Leaving the inhabitants with no neighbours to offer an awkward smile (or a resentful glance) to.
The reality differs, of course.
People live in few clusters, instead of occupying the entire landscape, mile by mile. The density that an Alaskan actually experiences is about 72 people per square mile.
That difference sits very comfortably in a position we all recognize as way off.
Now, let’s consider a conversation taking place in a hypothetical company, around SaaS metrics, and what unfolds as one hones the lens of reality.
Sam: We did $14.1K in MRR in this month.
Vaz: Oh that’s good, what did we do last month?
Sam: $13.9K.
Vaz: How much of that is new, though?
Sam: $10.5K
Vaz: Pitted against?
Sam: $10.7K, that we did last month.
Vaz: That’s not much of a win, is it? Okay. How about upgrades?
Sam: Those stand at $3.7K this month, as opposed to last month’s $3.2K
Sam: Almost the same. We lost $1.5K due to cancellations, last month it stood at $1.7K. Though, our free to paid conversion doubled because of a new campaign, and we made $4.8K, compared to last month’s $2.4K.
Vaz: That’s great. Why is the growth still marginal, though? What are we missing? Yes, what about downgrades?
Sam: Ah, let me check. This doesn’t look good.
Vaz: What is it?
Sam: We lost $3.2K in downgrades, tallied with $700 from last month.
Towards the end of the conversation, a chord is struck that announces the arrival of knowing what’s actually happening with the business.
Investigating further and closely observing the pattern of downgrades over the last few months, Sam and Vaz might learn about an over-selling of the product’s promise that took place a month or two ago.
A learning that can change how the organization operates, for the better.
Measures are vital. But when they’re kept unchecked from the thrust of reality, can confuse us, and can be easily willed to conform to certain ways, in order to please our notions of progress
A metric, alone, isn’t a business, just the way, a map isn’t a territory.
The message here is not to give up on maps. They can be wonderful and provide many useful insights. It is rather to understand their limitations. Each map carries the perspective of its creator and is limited by the medium it’s presented in. The more maps you have of a territory, the increased understanding you will have of the complexities of the terrain, allowing you to make better decisions as you navigate through it.
A good way to avoid being impaired by the single-track view of the world that a metric leaves us with, is to form our understanding based on multiple metrics, aimed at getting us closer to different parts of the truth.
The truth whose light plays on the vast surface of the sea of human interactions that an organization relies on. The truth that doesn’t just help us change perspectives, but behavior. The truth that lets us in on how little we know.
