I like frameworks because they provide structure and organization of thoughts around a particular topic. Lately the hot topic is generative AI. Before we dive into this topic and I propose a framework to use when thinking about the impact, let’s make sure we all are on the same page about generative AI, LLMs, GANs, etc. Generative AI (GAI) is a broad term that refers to any type of artificial intelligence that can create new content, such as text, images, audio, or video. Large Language Models (LLMs), such as ChatGPT, are a type of generative AI that are specifically designed to generate human-like text. Other types of generative AI include: Generative Adversarial Networks (GANs) often used by deepfake applications to generate faces talking, Variational Autoencoders (VAEs) which can be used to generate new images that are similar to existing images, and Recurrent Neural Networks (RNNs) that are often used to generate new music. The point being that there are a lot of AI techniques that can generate new content, LLMs being just one of them and they have been around for quite a while. The first GAI models were developed in the 1960s, but they were not very powerful. It wasn't until the advent of deep learning in the 2000s that GAI models started to become more powerful. Now, back to how we might organize our thoughts around this very broad and not so new topic of generative AI.
For tech companies I think there are three major areas to think about both opportunities and risks when it comes to GAI - engineering productivity, back office efficiency, and embedded into product offerings.
Engineering Productivity
Without getting into the difficulties of actually measuring software engineering productivity, it is generally accepted that for certain tasks GAI is a tool that can help increase an engineer’s productivity. Examples of this include providing natural language descriptions of code blocks, generating code from natural language descriptions, developing unit or integration tests, and automatically testing software for bugs. Leveraging this can free up engineers to focus on more creative and strategic work. Most teams that I’ve spoken to have started piloting some form of GAI for generating code from natural language descriptions. Some of the tools that can leveraged for this include:
Github Copilot is an AI-powered code completion tool that helps developers write code faster and sometimes more accurately. It is powered by OpenAI's Codex, a large language model that has been trained on a massive dataset of code. Github Copilot can generate code from natural language descriptions, complete incomplete code, and suggest alternative code implementations.
Google Cloud AutoML is a suite of machine learning services that helps developers build and deploy machine learning models without having to be experts in machine learning. AutoML includes tools for natural language processing, image recognition, and machine translation.
Amazon SageMaker is a fully managed machine learning service that helps developers build, train, and deploy machine learning models at scale. SageMaker includes tools for data preparation, model training, and model deployment.
From these pilots, I’ve heard estimates from 40% - 200% increases in productivity. I have also heard some engineers who are not super impressed. I suspect, like most things, the truth is in between and the key is to find the appropriate use for the tool. My experience has been that with very simple coding tasks, these tools perform well and are very fast. For complex tasks that require a lot of context they don’t perform well. I also suspect this will continue to improve until its use is as prevalent as IDEs (integrated development environment) such as Visual Studio or Eclipse and just us curmudgeons will continue to use VIM and shake our fists at how it’s all changed. (I’m kidding of course about VIM since a reported 23.34% of professional developers still use it regularly.)
From a recent thread on Stephen Wolfram’s interview with Lex Fridman on How ChatGPT changes the future of programming there were several interesting comments such as:
I don’t write code anymore without using some AI tool like copilot are [sic] chatgpt to answer questions. I have found that it doesn’t make programming easier, it actually makes it harder. Because the AI does all the easy stuff and I am only left with the harder things to figure out.
…
However, if you treat it like an intern or junior programmer and prompt it how to improve the code it does usually improve to quite a useable form. And the fact it does it seconds after your prompt makes for very fast iteration time.
As impressive as the possible productivity increases are from code automation, I think the biggest impact to software development from GAI might be in automating testing. While there is still debate on whether automated testing improves code quality, one hypothesis is that the problem resides with code coverage. In research presented at the 2015 30th IEEE/ACM International Conference on Automated Software Engineering, researchers stated, “Our experiments indicate that in many cases, code coverage remains a major problem, preventing automatically generated unit tests from detecting faults.” Most places with any significant sized code base struggle to get code coverage above 50%. GAI might help get this to close to 100%.
Back Office Efficiency
Almost every business has some form of back office work even if it is outsourced. Companies need functions such as HR, legal, and finance for everything from reviewing contracts to closing the books. Automation has been employed for years to help with many more mundane tasks in these functions. The rise of GAI can exponentially increase this for tasks such as generating contracts, processing invoices, and managing employee benefits. This can free up employees to focus on more strategic work, or shift those roles closer to the customer such as product, engineering, or marketing.
In addition to automating tasks, generative AI can also be used to improve the quality of work in back office functions. For example, generative AI can be used to generate more accurate and up-to-date reports, or to identify potential risks and problems such as data that is missing or possibly inaccurate. This can help to improve the efficiency and effectiveness of back office operations.
According to EDRM, a global e-discovery company, legal teams can leverage GAI in ways ranging from client communication to education to document drafting to contract reviews. The legal bot DoNotPay claims to “Fight corporations, beat bureaucracy and sue anyone at the press of a button.” I don’t suspect that corporations will soon leave their legal representation up to a bot or LLM, but I do think small startups are going to continue to leverage this tech more and more instead of paying $500+ per hour attorney fees. A CEO I spoke with recently has a goal of reducing back office expenses more than 50% this year by leveraging GAI tools.
Product Offerings
Beside productivity and efficiency gains, GAI can also be used to improve the functionality and user experience of products. For example, generative AI can be used to generate personalized recommendations, create more engaging content, and improve search results through conversational interfaces. VentureBeat reported on a Juniper study that claims, “AI-powered chatbots are predicted to play a significant role in customer interactions over the next few years, handling 70% of customer conversations in 2023.” These types of integrations will help to make ecommerce search and discovery functionality simpler and more natural, allowing shoppers to find products faster.
These applications are also not new ideas. A lot of recommendation systems have been using GAI for a while. Google Search uses GAI to improve the relevance of its search results. It does this by generating personalized search results based on a user's past search history, interests, and location. Netflix uses GAI to recommend movies and TV shows to its users. They do this by using past viewing history, ratings, and reviews to generate recommendations that are similar to a customer’s viewing history. In addition to improving the functionality of product offerings, GAI can also be used to improve security. For example, GAI can be used to identify potential scam products on marketplaces, or in security features to protect PII.
Generative AI is a powerful tool with the potential to revolutionize many industries; however, that revolution started years ago without much fanfare. With the public launch of ChatGPT, OpenAI shined a light on GAI that makes it almost impossible to ignore. Whether or not you agree with my thoughts on how GAI might help engineering productivity, back office efficiency, or product offerings, a framework such as this can be helpful to organize your thoughts and then form a strategy about how to leverage it or protect from it.