A mental framework for thinking about where to properly apply metrics
Excellent piece. Any form of measurement can be seen as setting an incentive, and nearly every incentive can lead to unintended consequences. I recall a conversation during a casual meetup over coffee when an engineer, after a few drinks, boasted about writing fewer than 10 lines of code a year and still drawing a six-figure salary. Essentially, he felt he was being paid to do next to nothing since the company's executives emphasized maintaining system stability by tracking Mean Time Between Failures (MTBF). In essence, if you implemented a change that caused the website to crash, you'd lose your bonus. From an engineering standpoint, the most foolproof way to maintain a flawless MTBF metric is to resist making changes. Introducing such a Key Performance Indicator (KPI) can inadvertently encourage undesired behaviors, such as avoiding code modifications. Jokingly, I asked him how he'd react if the company started monitoring his coding activity. His reply was both disheartening and amusing: "Why not just sprinkle in a few unit tests or add redundant code that's never activated?"
Ultimately, the most reliable gauge of engineering quality might be peer evaluation. Interestingly, during my time as an individual contributor, it was always evident who was genuinely adding value (whether through code reviews or other interactions) and who was merely coasting.
But the challenge remains: What happens when you onboard an enthusiastic fresh graduate eager to contribute, but who, due to inexperience, inadvertently destabilizes the system? Almost all contemporary software systems are intricate, often running into thousands of lines of code.
From a business standpoint, it's easy to see the challenges. Research & Development can be pricey, and determining its return on investment is no simple task. In the end, all measurement mechanisms have their flaws, though some can be helpful temporarily. This underscores the importance of tech-savvy leadership. Drawing from your surgeon analogy, managing a hospital might be more straightforward if the person at the helm possesses a solid medical background.