Unintended Consequences

The perils of poorly designed metrics and incentives

It started with a man named Alphonse, born on a cold January day in Brooklyn, New York. The year was 1899. There was nothing particularly unusual about Alphonse Capone. He grew up in a neighborhood where the smell of fresh bread wafted through the streets, and children ran barefoot on sidewalks rough with grit. Al was like most kids, playing stickball in the alley and staying out until the recently invented street lights flickered on. He even went to school, at least for a while, until he punched a teacher and was expelled at fourteen. After that, the streets became his classroom.

The streets had a different curriculum. Al learned how to hustle, how to smile with a broken nose, and how to strike a deal in a darkened alley. He became a bouncer in a rough joint in Coney Island, where he got his trademark scar, a slashing cut to the face from a disgruntled patron. That's how Alphonse Capone became "Scarface," though he preferred when friends called him simply "Al." Eventually, he found his way to Chicago, taking orders from Johnny Torrio, a business-minded crime boss who saw potential in the young Capone. Al was smart, ruthless, and loyal, all the right ingredients for a man Torrio could trust.

Chicago was a city of dreams and dust, smoke and ambition. It was here that Al found his true calling. It was 1920, and America had embarked on a grand experiment, Prohibition. The Volstead Act had been passed, making the manufacture, sale, and transportation of liquor illegal. The moralists were sure that Prohibition would cleanse society, make America a safer, more virtuous land. But they forgot one thing: where there is demand, there will always be supply. And Al Capone? He was more than happy to supply.

Capone built an empire on the back of that thirst. Speakeasies sprung up like dandelions in the cracks of Chicago's sidewalks, and Capone made sure they all had what they needed. He became the kingpin of a booming underground industry, raking in millions while the rest of the country struggled with the Great Depression. By the late 1920s, he was the undisputed boss of Chicago, ruling by the iron law of supply and demand. People wanted a drink, and Al Capone was the man who could get it for them.

The irony is this: the very law that was meant to end crime had, in fact, created it. Prohibition wasn't just the downfall of the nation's morals, it was the making of men like Al Capone. It turned petty criminals into titans of the underworld and gave rise to organized crime that would haunt America for decades to come. The cure had been worse than the disease. And that is the theme of today’s newsletter…unintended consequences of decisions.

The French cultural theorist Paul Virilio is famous for his quote,

When you invent the ship, you also invent the shipwreck; when you invent the plane you also invent the plane crash; and when you invent electricity, you invent electrocution...Every technology carries its own negativity, which is invented at the same time as technical progress.

This quote serves to remind us that any technological invention, and I would argue any decision, has a potential downside. Newton’s third law of motion, “For every action there is an equal and opposite reaction" seemingly also applies to this more ethereal space of technology and human thoughts. When we make a decision, take an action, or even put a metric in place to incent an individual or team performance, we are likely to create unintended consequences and behaviors. Take Wells Fargo’s sales metric program as an example. 

In the early 2010s, Wells Fargo implemented aggressive sales quotas that pressured employees to open new customer accounts. The metric-driven targets were intended to boost growth and solidify the bank's market position, but the outcome was anything but positive. Employees, fearing repercussions for missing quotas, began opening millions of unauthorized accounts without customer consent. Fake credit cards, checking accounts, and other financial products were created, ultimately leading to widespread fraud that went undetected for years.

When the scandal broke in 2016, it caused a public uproar, resulting in significant reputational damage and regulatory penalties for Wells Fargo. The company paid billions in fines, and several top executives, including the CEO, resigned under pressure. The incident highlighted the dangers of focusing too heavily on metrics without considering the potential downsides or unintended consequences of these measurements.

Hopefully not to this extent, but in product development organizations, we often see metrics that, when solely focused on, cause unintended consequences. As a hypothetical example, if a squad is incentivized to increase advertising revenue, they might make changes to drive up ad revenue but accidentally decrease customer engagement. By including counterbalancing metrics, such as measuring user engagement or satisfaction alongside ad revenue, organizations can maintain a balance that drives both revenue and positive customer experiences. To avoid these kinds of unintended consequences, it is crucial to implement counterbalancing metrics that provide a more holistic view of performance. For instance, Wells Fargo could have included customer satisfaction or retention metrics alongside account creation targets to ensure the focus remained on ethical, customer-centered growth.

The “Cobra Effect” is another tale of caution about unintended consequences of incentives. In the late 19th century, during British colonial rule in India, the government faced a growing cobra population in Delhi. To tackle this problem, they announced a bounty for every dead cobra brought to the authorities. Initially, the plan appeared successful, as many cobras were killed and people collected the rewards. However, some individuals saw an opportunity to profit further by breeding cobras instead of hunting them. In backyards and hidden sheds, cobras were hatched and raised, only to be killed for the bounty. The snake population didn't decrease as expected, but the payments continued to flow out of the government coffers.

Eventually, the authorities discovered the scheme. Outraged by the exploitation of their program, they promptly canceled the bounty. With no incentive to keep their cobras, the breeders released the now-worthless snakes into the city, leading to an even larger cobra population than before. This unintended consequence, later known as "The Cobra Effect," serves as a reminder that poorly designed incentives can lead to the opposite of the desired outcome. The British had tried to solve a problem, but their solution ultimately made it worse, demonstrating that the road to good intentions is often paved with unforeseen complications.

In business, and particularly in technology and product development, it’s tempting to simplify success into a few key performance indicators (KPIs). Metrics can be powerful tools to guide behavior and focus efforts, but they are double-edged swords. The danger lies in thinking they tell the entire story. Metrics can distort priorities, incentivize bad behavior, or cause teams to lose sight of what really matters: the humans behind the numbers. We have two eponymous laws that cover this:

Goodhart’s Law -  When a measure becomes a target, it ceases to be a good measure

Campbell’s Law - The more a metric is used for decision-making, the more it will be subject to corruption 

In product development, avoiding these pitfalls requires not only defining the right metrics but also maintaining a willingness to revisit and refine them. Metrics should not be static; they must evolve as the context and the goals change. Moreover, leaders must remain vigilant to ensure that the pursuit of those metrics does not lead to a loss of purpose or values. Sometimes it means slowing down, adding friction, or choosing a less profitable path because it’s the right one.

One of the most effective tools to counteract unintended consequences is building a culture that values reflection and critical thinking. It's not enough for product managers or engineers to focus on the "how" of building features; they must also constantly ask "why." Why are we incentivizing this behavior? Why do we think this metric will drive the right outcomes? And importantly, what are we not seeing? Encouraging these questions helps surface potential negative impacts before they manifest in costly ways.

It's easy to fall into the trap of thinking that numbers are neutral, that they are simply representations of reality. But metrics are inherently value-laden. What we choose to measure reflects what we value, and how we measure it influences behavior. The unintended consequences arise not because the numbers are wrong, but because we too often ignore the complexity of the systems we’re trying to shape. We optimize for parts without seeing the whole, forgetting that for every ship, there is also a shipwreck.

As product developers, business leaders, or even as individuals, we must approach our work with a sense of humility. It is not enough to simply focus on technical progress or growth metrics; we must also consider the broader social implications and potential downsides of our decisions. Like the moralists who thought Prohibition would create a virtuous society, we often assume our intentions will yield positive results. But history, from Al Capone to Wells Fargo, reminds us that intentions are not enough. We must be deliberate, thoughtful, and willing to adapt when faced with the unintended consequences that inevitably arise.