Hi there,

Have you ever used a ladder with a broken rung? You're a little scared at first, but you can easily skip the shaky step and still effectively use the ladder. With growing ease, you continue using it until you fall one day. 

Because you didn't fall the first time, you figured that using a broken ladder was safe. Sounds logical enough, right? No. The decision to climb a broken ladder was always a risky one - a favorable outcome merely paints it in a better light. If you had fallen the first time, it would have been out of your house before you could say "outcome bias." 

For all of us, judging something or someone is often a split-second decision. Outcome bias is a cognitive bias that evaluates our decision-making based on the outcome rather than the quality of the process itself. 

Consider a doctor's choice to offer a patient a heart bypass: It adds several years to the patient's life, with a small risk of death. We're more likely to blame the doctor's decision if the patient died than in a scenario where the patient lived. Duh, right? The risks and benefits were identical in both scenarios. 

Conversely, faulty decision-making that results in a positive outcome goes uncorrected. Near misses are treated as close calls that could have worsened and not indicate an impending disaster. We heave a sigh of relief and trudge on. 

In 2003, NASA's Columbia Space Shuttle disintegrated upon re-entry to Earth, killing all seven astronauts on board. The cause: A piece of insulating foam from the external tank hit one of the orbiter's wings. This had occurred on every Space Shuttle flight begging the question - Why did NASA not fix this obvious design flaw? 

The team started seeing the debris as an acceptable risk with each successful landing. On that fateful mission, the most significant piece of foam hit the orbiter, causing the shuttle to burn up on re-entry. 

In organizations, the outcome bias can look like increased confidence in your "gut instincts" after a favorable outcome or the scrapping of experiments after one with a bad result. Fighting the outcome bias can be a cognitive strain; you should question successes rigorously as you dissect failures. 

Companies today are constantly taking chances and trying bold things. In a culture of constant rapid experimentation, have you ever thought about how your organization assesses the success of an initiative? Take a closer look at the things that work and the things that don't - the underlying causes might surprise you! 

In this week's Compass, we're looking at why and how you can leverage experimentation in different industries.

Have you encountered any instance of outcome bias at your workplace? Share your stories; let's unlearn together!

How you can build an efficient experimentation engine within your company

The what, why, and how of building an innovation culture

Read More >

 

A guide for e-learning businesses to experiment your way to hypergrowth

Do you have the right tools to help you experiment at scale? 

Read More >

 

A teardown of SaaS Pricing Experiments: Shopify, Zendesk, and StatusPage

Each product demands a different approach. Can you find yours? 

Read More >

Read the numbers right: Learn how to expertly navigate SaaS analytics and understand your subscription business effectively. This online course can help you identify causal relationships among your core metrics, how they evolve depending on what business stage you’re in, and how you can align your metrics to your business goals.  Register now>