Cultivating a Culture of Excellence

Goals, Metrics, Experimentation, and more

This post is a collaboration by Tyler Folkman, Chief Technology and AI Officer at BENlabs, and Mike Fisher, advisor and former CTO at Etsy, and is being posted on Learning With Data and Fish Food for Thought. 

In the modern business world, companies are constantly faced with risks and competitive challenges that we have to address. In this type of environment, driving product innovation in order to maintain a competitive edge requires more than just a capable team. It requires a culture of excellence that enables teams to create exceptional products, fosters a positive work environment, and ultimately drives the organization forward. There are many factors involved in creating a culture of excellence. In this post, we’ll focus on a few that we believe are the most important factors but there are of course many others as well. 

Empowering Teams

Empowering your engineering and product teams is a key ingredient to foster a culture of excellence. Marty Cagan, a thought leader in technology product management, emphasizes the significance of empowerment in his book, Empowered: Ordinary People, Extraordinary Products. He and his co-author Chris Jones state, "...in strong product companies, teams are instead given problems to solve, rather than features to build, and most important, they are empowered to solve those problems in the best way they see fit. And they are then held accountable to the results.”

In contrast to empowered teams, we’ve all seen product engineering teams who are task driven. They are told what to do by some stakeholder, usually the CEO or a sales representative, who thinks they know exactly what next feature is needed. We refer to these types of teams as order takers. The team is given an order and asked to execute on it, usually by a certain date. If this is how your teams work, you are not taking advantage of the value of product management. True product management is a force multiplier, which are tools, elements, or a combination of factors that dramatically increase the effectiveness of a group. A great product manager makes all of the team - engineers, designers, analysts, etc. - more efficient and productive. 

Empowerment means more than assigning tasks and hoping for the best. It means entrusting teams with real authority, accountability, and autonomy. It means setting clear expectations, then stepping back to let your teams innovate and create. It involves fostering an environment where failure is seen as a stepping stone to success, and where the freedom to experiment is encouraged. This creates a sense of ownership and pride, driving teams to deliver their best work. Empowerment is intricately related to proper metrics, which we’ll talk about next. 

Measuring Results: Metrics That Matter

One of the most critical parts of building a culture of excellence is establishing effective measurement systems. The key is to focus on metrics that truly matter to your business. The concept of “vanity metrics” was first introduced in the book Lean Startup launched by Eric Ries in 2011. It's easy to be lured into tracking vanity metrics – traffic or page views, etc. – because they are fun to watch go up and to the right on graphs. However, they often don't correlate with your organization's overall success. In fact, sometimes these metrics cost us more than they make in revenue. Traffic that bounces without conversions usually costs more than it’s worth. 

Tableau, a data visualization software company focused on business intelligence, states on their website that, “Vanity metrics are metrics that make you look good to others but do not help you understand your own performance in a way that informs future strategies. These metrics are exciting to point to if you want to appear to be improving, but they often aren’t actionable and aren’t related to anything you can control or repeat in a meaningful way. Vanity metrics are most often contrasted against actionable metrics, which is data that helps you make decisions and helps your business reach its goals or grow.”

Another false metric that teams get tricked into celebrating is the launch or delivery of a feature. While celebrating the delivery is fine, you should make a point that this is just an intermediate step on the way to solving the customer problem that you set out to accomplish. Often this requires iteration on the feature. In my experience, it is not uncommon for 85% of new features to be either neutral, they did nothing to the customer experience, or actually negative, in that they hurt the customer experience. A Harvard Business Review article from 2017 states, “At Google and Bing, only about 10% to 20% of experiments generate positive results. At Microsoft as a whole, one-third prove effective, one-third have neutral results, and one-third have negative results.” Launch, measure, iterate, and once you’ve accomplished the goal, then celebrate.

In contrast to vanity metrics or date-based launch metrics, measuring outcomes such as revenue growth, customer churn, conversion rate, etc. provide a more realistic picture of your product's impact and your team's performance. It is true that almost every metric can be a vanity metric if you are not careful but starting with core financial metrics such as revenue and costs are solid foundational metrics. 

Hiring and Firing

We all enjoy bringing new members onto our teams. Hiring new employees, while demanding in terms of time and energy, is usually a positive sign that the company is doing well and the team is performing. In contrast, none of us enjoy having to fire an employee, but it’s just as much part of building a culture of excellence as hiring is, unfortunately. Hiring and firing are both crucial elements of building a high-performance culture. They are not just administrative tasks, but strategic decisions that significantly impact the organization's future. A helpful approach to these processes is the "Seed, Feed, Weed" analogy. While people are certainly much more than a flower or vegetable in our garden, we can use this analogy to think about the different steps that are equally important. 

Seed: The first step in creating a culture of excellence or terrific garden, involves planting seeds. This involves identifying and hiring high potential individuals who are aligned with your organization's vision, values, and culture. They are seeds that, when planted, can sprout into a thriving component of your organization's success. As a hiring manager, you should be thinking about seeds that you plant as ideas that attract excellence. This could be the coffee meetings that you take with people that you know are really excellent, even when you don’t have a job opening and they aren’t looking. This could be posts on social media about the cool things your team is doing or comments on other people’s posts. You are planting the seed that someday when the timing is right, might blossom into a hiring opportunity. 

Feed: Once these seeds are planted, they need nurturing or "feeding" to grow. This involves providing the necessary tools, training, and resources to enable these individuals to thrive. Feeding also includes providing the right environment – a positive work culture, a supportive team, and an opportunity to innovate and feel empowered. Feeding doesn’t just mean promoting them. As Christian Idiodi from SVPG once said, “Don’t promote people into incompetence, coach them.” Be an excellent mentor, find challenging roles or temporary assignments for them. At each review, don’t just tell them a competency they need to work on, sit down with them and determine how you can help them develop that competency through training, coaching, or roles. 

Weed: Lastly, comes the painful but essential process of weeding. While we generally think of this step as letting someone go from the team it actually involves many more steps long before that’s necessary. The best advice for this part of the process is the Golden Rule, treat them like you’d want to be treated. This usually means being very upfront as soon as their behavior or performance isn’t measuring up. Being overly nice and not being direct is not helpful and not how you’d want to be treated. Be direct and document everything. Miscommunications are easy to occur, especially during stressful conversations such as performance reviews. Write down what you are going to say, use it as speaking notes during the conversation, and send the individual a note summarizing what you said. If you’ve tried and things aren’t improving after a reasonable period of time, take action quickly but in a human-centered way. Provide the individual with as much cushion for a soft landing as possible. This might involve severance, it might involve employment coaching, it might even involve reference to positions where they can really succeed. 

Team Dynamics

While hiring and firing are absolutely critical, an insight from Google that was shared in the book Accelerate was, “Who is on a team matters less than how the team members interact, structure their work, and view their contributions.” It all comes down to team dynamics. It is so much easier to believe that you can go and hire a few “10x” engineers and ride to victory. The truth is, though, that building products is a team sport. And while, as mentioned previously, who is on your team matters a lot. You can’t ignore the team dynamics.

Fortunately, team dynamics is something over which leaders can have a lot of control and influence. The three areas of team dynamics that can have a lot of leverage are: trust, failure, and shared responsibility.

Trust is the foundation of strong team dynamics. In their study of teams, Google found that psychological safety was the number one most important factor within team dynamics and trust is the key to building that safety. Trust is what enables people to bring all their talents, thoughts, and ideas to your team without fear. Trust is what allows people to speak up when they think the team could be better. Trust means the team will rely on each other to solve the many challenges they will encounter.

Failure. We all know it's important and yet none of us like it. How your teams handle failure is probably the easiest way to understand its health. When a team fails, do they immediately start pointing fingers at each other and look for someone to blame? Or rather, do they come together and view failure as a challenge to overcome on the road to success? Healthy teams know failure is part of the game and those failures make them better, stronger, and excited to improve.

Lastly, shared responsibility. Since building great products is a team sport, your team needs to win and lose as a team. They need a shared goal and they need to view the accomplishment of that goal as everyone’s number one priority. While individuals on the team might have personal improvement goals, they shouldn’t have any other business goals than the goal of the team. Everyone should do whatever they can to help the team win.

Velocity, Experimentation, and Learning

Velocity might be the most important goal of any company. By velocity, I mean the speed at which you can make bets that you believe will create value for your customers. The faster you are at delivering these bets, the less you have to be right, the quicker you learn, and the more you can experiment.

If you look at the history of how teams delivered these bets we have gone from a Waterfall method in which a significant time is spent up front to detail exactly what to build, to Agile methods in which teams strive to deliver value more iteratively and faster. A few reasons Agile became a powerful method was that many more of the problems we needed to solve were complex (many unknowns) and we had more tools and technology to be more nimble.

At the end of the day, though, every process strives to deliver the most value to customers as quickly as possible. That is the principle. Thus, in order to have excellence in your culture you need to first understand how well you do at delivering value quickly and then always strive to improve.

To understand, I would suggest starting with two metrics: how quickly you can deliver to customers and how often you are correct in the value you anticipated being created. A common metric to track speed of delivery is cycle time which is usually measured as the time from first commit to production release. But as we have all learned, just delivering things is not the goal. The goal is to create customer value, so you also need to track the fraction of bets you make that created the anticipated value.

If you start tracking these metrics and find you are not doing particularly well with either, start with improving the speed at which you can deliver. Nothing happens if you don’t get new features or experiences into your product. Also, the faster you are at just delivering any type of bet, the more bets you can learn from to understand what is and isn’t valuable and the less scared you’ll be of experimenting because the time to deliver will be less.

Clear Goals and Alignment

The book Playing to Win has a statement that has always resonated: you need to play to win because winning is so hard that it's extremely unlikely you will win by accident (paraphrase). Building a successful product could be one of the hardest endeavors out there, so you better be playing to win. One of the biggest ways teams don’t play to win is by not having clear goals and alignment. It is sort of like that phrase, “If you don’t know where you are going, you likely won’t get there.”

One of the most popular ways to set clear goals and alignment for a team is OKRs. You don’t have to use OKRs, but the principle behind them is pretty solid. Every team should have a clear objective - the team needs to know what it means to win. Not only do they need to know what it means to win, they all have to have the same clarity on what that vision looks like. So write it down! Have the team spend some time with it. Make sure everyone on the team clearly knows what the objective is and share the same vision. 

Second, the team needs to know how they will be evaluated. OKRs suggest you use key results to accomplish this task. Key results can be extremely powerful because they are a clear scorecard, but getting the key results figured out can be tricky. It can be hard to know what the right key results are and when you find them you might struggle to measure them effectively or in a timely manner. 

To get to great key results, focus on input metrics that you know, or at least currently believe, are causal to output metrics you care about and are mostly within the team’s control. For example, if you want to improve your day one conversion rate, that is really more of an output metric that is lagging. An input metric might be the average time it takes a new user to take a core action within your product. The underlying belief for that to be an input metric would be that doing a core action as quickly as possible causes more people to convert. This input metric is significantly more within the control of the team and will likely lead to changes in your output metric. And if it doesn’t, you’ve now learned that the casualty you thought existed, doesn’t.

We hope these factors help you as you drive towards a culture of excellence. Building products that people love is likely one of the hardest endeavors one can pursue. The path is filled with unknowns, uncountable details to consider, and requires an almost inhuman amount of dedication and focus. In our experience, though, fostering a culture of excellence is one of the highest leverage activities for a leader. As you work to apply these concepts to your teams, continue to strive for excellence and you’ll be amazed at what you can achieve.