Quality vs Quantity

More opportunities to learn lead to better outcomes

The following is a story told by James Clear in his book Atomic Habits. 

In the initial session of a photography course at the University of Florida, Professor Jerry Uelsmann segregated his students into two distinct categories.

The group on the classroom's left side was assigned to the "quantity" division. Their grading would be based on the sheer volume of their photographic output. On the course's final day, a tally of each student's submitted photos would determine their grades: 100 photos for an A, 90 for a B, 80 for a C, and so forth.

On the opposite side, the right-hand students were placed in the "quality" category. Their assessment would focus exclusively on the caliber of their work. Producing just a single photo for the semester was required, but for top marks, this photo had to be near flawless.

At the term's conclusion, a surprising trend emerged. The top photographs all originated from the quantity-focused group. These students spent the semester actively engaged in shooting, experimenting with various elements like composition and lighting, and refining their techniques through trial and error in the darkroom. Their extensive photo production significantly sharpened their skills. Conversely, the quality-focused group spent most of their time theorizing about the perfect shot, resulting in minimal practical work and generally unremarkable final photos.

You might have come across this story in other articles and books as it has been repurposed and retold many times. One distinction by many authors is that the object in the story is often changed to something more tangible such as ceramic pots like the authors David Bayles and Ted Orland did in their book Art & Fear. The wonderful thing about this story is that it can be repurposed. This story can serve us in many domains as an example of how when we forget about trying to deliver a quality product and instead focus on learning through trial and error, we can often achieve the highest quality products.

When I read this story, I immediately thought of product development. One way to develop a product, which seems intuitively the ideal way but actually isn’t, is similar to how we might build a house. Start by focusing on what features you want built, prepare the scope document, estimate how long it will take and how much it will cost, and then get started building. The goal is to build as high of a quality product as possible as measured by the scope, cost, and time. The fallacy here is that those aren’t the true measures of quality. The real measure should be from the perspective of the customer using the product. Does it solve their problems, is it a joy to use, are they ultimately satisfied? 

As we’ve all experienced with projects planned this way, there is always some amount of budget and schedule overrun. What most contractors or project managers do is overestimate by some percentage. This gives them a buffer to expand into and still come close to the timeline and budget. Does this produce the highest quality results? Well, not really. Eventually we have a house and we’re hopefully satisfied with the product because the contractor is on to their next project. What I hear from most people who undertake such a project is that they learned a lot by living in the house and often, if given the chance, would change some things. The goals of completing the project within a scope, at a certain cost, and in a specific time, aren’t the right goals. Also, the process of doing it in one big launch isn’t ideal.

There is a better way to build products, especially digital products that are much cheaper to iterate on. This better way starts with the right goals. We should be focused on solving a customer problem that will result in improving a business metric such as increased revenue. We should not be thinking of a goal in terms of delivering a product or feature. Doing so misses the point of the product or feature, to solve a customer problem as measured by a business metric. It is common for 85% of new features to be either neutral — making no difference to the customer experience, or negatively impacting it. A 2017 Harvard Business Review article reported that, "At Google and Bing, only 10% to 20% of experiments yield positive results. At Microsoft, a third are effective, a third neutral, and a third negative." If your goal is to deliver a feature you are 85% likely to not have solved any customer problem or made their experience better. 

Starting with the goal of solving a customer problem as measured by a business metric puts us in the right mindset. Then we use a process of iteration on small incremental changes to see what works and what doesn’t. Often this is done with A/B testing, to see statistically if we’re having the right impact on our customers. Every experiment is a learning opportunity. Just like the photography students in the opening story, our product development teams should be focused on experimenting and learning. By doing so they will end up with the highest quality product as determined by customer and business satisfaction. 

We’ve all seen projects that required heroic efforts to come in on time, on budget, and complete scope only to fail in the marketplace. In the worst case, despite heroic efforts, the project misses on budget, schedule, or scope and fails with consumers. Take Windows Vista, originally planned for a 2003 release, it finally launched in 2007. It is often remembered for its ambitious but flawed attempt to significantly upgrade Windows' security and visual appeal, which ultimately fell short in user experience and performance. 

The argument that focusing on quantity over quality is more beneficial can be further strengthened by considering the nature of learning and innovation in product development. Just like the photography students who improved through constant practice, product development teams benefit from iterative processes and frequent experimentation. Iterative development allows teams to respond to customer feedback, adapt to changing market conditions, and test new ideas quickly. This approach contrasts with the traditional method of aiming for a perfect product in one go, which often leads to missed opportunities for improvement and adaptation.

By prioritizing the quantity of opportunities to learn, teams encourage a culture of continuous learning and flexibility. Each iteration becomes a learning opportunity, just as each photograph was for the students. This method aligns more closely with real-world conditions where customer preferences and market dynamics are constantly evolving. Furthermore, this approach reduces the risks associated with big launches like Windows Vista, where a significant investment is made in a single, large-scale product release. Instead, smaller, more frequent releases allow for adjustments and refinements based on actual user feedback and engagement.

Start by ensuring you have the right goals – business and customer outcomes, not feature launches. Then as a process, focus on quantity – iterative, continuous improvement, and more learning opportunities. These two together lead to higher quality outcomes that meet actual customer needs and business goals. This fosters a more agile and responsive approach to product development, ultimately resulting in products that better meet customer needs and succeed in the market. The next time you want to build a high quality product, think about focusing on the right goals and the quantity of learning opportunities that you can create and let the quality come naturally.