For all the benefits that generative AI promises, voices are getting louder about the unintended societal effects of this technology. Some wonder if creative jobs will be the most in demand over the next decade as software engineering becomes a commodity. Others worry about job losses, which may necessitate reskilling in some cases. It is the first time in humanity’s history that white-collar jobs stand to be automated, potentially rendering expensive degrees and years of experience meaningless.
But should governments hit the brakes by imposing regulations or continue to improve this technology, which will completely change how we think about work? Let’s explore:
Generative AI: The new California Gold Rush
The technological breakthrough that was expected in a decade or two is already here. Not even the creators of ChatGPT expected their creation to be this wildly successful so quickly.
Compared to some technology trends of the last decade, the critical difference here is that the use cases are real, and enterprises have budgets already allocated. This is not an excellent technology solution that is looking for a problem. This feels like the beginning of a new technological supercycle that will last decades or even longer.
For a long time, data has been referred to as new oil. With a large volume of exclusive data, enterprises can build competitive moats. To do this, the techniques to extract meaningful insights from large datasets have evolved over the last few decades from descriptive (e.g., “Tell me what happened”) to predictive (e.g., “What should I do to improve topline revenue?”).
Whether you used SQL-based analysis spreadsheets or R/Stata software to complete this analysis, you needed more regarding what was possible. However, with generative AI, this data can be used to create entirely new reports, tables, code, images, and videos in seconds. It is so powerful that it has taken the world by storm.
What’s the secret sauce?
At the basic level, let’s look at the simple equation of a straight-line y=mx+c.
This is a simple 2D representation where m represents the slope of the curve and c represents the fixed number, which is the point where the line intersects the x-axis. In the most fundamental terms, m and c represent the weights and biases, respectively, for an AI model.
Now, let’s slowly expand this simple equation and consider how the human brain has neurons and synapses that work together to retrieve knowledge and make decisions. Representing the human brain would require a multidimensional space (called a vector) where infinite knowledge can be coded and stored for quick retrieval.
Imagine turning text management into a math problem: Vector embeddings.
Imagine if every piece of data (image, text, blog, etc.) could be represented by numbers. It is possible. All such data can be represented by a vector, just a collection of numbers. You get something called embedding when you turn all these words/sentences/paragraphs into vectors and capture the relationships between different phrases. Once done, you can turn search and classification into a math problem.
In such a multidimensional space, when we represent text as a mathematical vector representation, we get a clustering where words that are similar in meaning are in the same cluster. For example, in the screenshot above (taken from the Tensorflow embedding projector), words closest to the word “database” are clustered in the same region, making responding to a query that includes that word very easy. Embeddings can be used to create text classifiers and to empower semantic search.
Once you have a trained model, you can ask it to generate “the image of a cat flying through space in an astronaut suit,” it will create that image in seconds. For this magic to work, large clusters of GPUs and CPUs run nonstop for weeks or months to process the data the size of the entire Wikipedia website or the entire public internet to turn it into a mathematical equation where each time new data is processed, the weights and biases of the model change a little bit. Such trained models, whether large or small, make employees more productive and sometimes eliminate the need to hire more people.
Competitive advantages
Do you/did you watch Ted Lasso? Single-handedly, the show has driven new customers to AppleTV. It illustrates that to win the competitive wars in the digital streaming business, you don’t need to produce 100 average shows; you need just one that is incredible. In the world of generative AI, this happened with OpenAI, which had nothing to lose as it kept iterating and launching innovative products like GPT-1/2/3 and DALL·E. Others with deeper pockets were probably more cautious and are now playing a catchup game. Microsoft CEO Satya Nadella famously asked about generative AI, “OpenAI built this with 250 people; why do we have Microsoft Research at all?”
Once you have a trained model to which you can feed quality data, it builds a flywheel, leading to a competitive advantage. More users are driven to the product, and as they use it, they share data in the text prompts, which can be used to improve the model.
Once the flywheel above of data -> training -> fine-tuning -> training starts, it can be a sustainable competitive differentiator for businesses. Over the last few years, vendors, both small and large, have been maniacal about building ever-larger models for better performance. Why would you stop at a ten-billion-parameter model when you can train a massive general-purpose model with 500 billion parameters that can answer questions about any topic from any industry?
Recently, we realized that we hit the limit of productivity gains that a model’s size can achieve. You might be better off with a smaller model trained on precise data for domain-specific use cases. An example would be BloombergGPT, a private model trained on financial data that only Bloomberg can access. It is a 50 billion-parameter language model trained on a massive dataset of financial articles, news, and other textual data they hold and can collect.
Independent evaluations of models have proved that there is no silver bullet, but the best model for an enterprise will be use-case-specific. It may be large or small; it may be open-source or closed-source. The comprehensive evaluation completed by Stanford using models from openAI, Cohere, Anthropic, and others found that smaller models may perform better than their larger counterparts. This affects the choices a company can make regarding starting to use generative AI, and there are multiple factors that decision-makers have to take into account:
Complexity of operationalizing foundation models: Training a model is never “done.” It is a continuous process in which a model’s weights and biases are updated each time it goes through fine-tuning.
Training and inference costs: There are several options available today, which can each vary in price based on the fine-tuning required:
Train your model from scratch. This is quite expensive as training a large language model (LLM) could cost as much as $10 million.
Use a public model from a large vendor. Here, the API usage costs can increase rather quickly.
Fine-tune a smaller proprietary or open-source model. This has the cost of continuously updating the model.
In addition to training costs, it is important to realize that each time the model’s API is called, the costs increase. For something simple like sending an email blast, if each email is customized using a model, the cost can increase up to 10 times, thus negatively affecting the business’s gross margins.
Confidence in wrong information: Someone with the confidence of an LLM has the potential to go far in life with little effort! Since these outputs are probabilistic and not deterministic, the model may make up an answer once a question is asked and appear very confident. This is called hallucination, and it is a significant barrier to the adoption of LLMs in the enterprise.
Teams and skills: Over the last few years, I have talked to numerous data and AI leaders, and it has become clear that team restructuring is required to manage today’s massive volume of data companies deal with. While using case-dependent to a large degree, the most efficient structure is a central team that manages data, leading to analytics and ML analytics. This structure works well not just for predictive AI but also for generative AI.
Security and data privacy: It is so easy for employees to share critical pieces of code or proprietary information with an LLM, and once shared, the data can and will be used by the vendors to update their models. This means that the data can leave an enterprise’s secure walls, which is a problem because, in addition to a company’s secrets, this data might include PII/PHI data, which can invite regulatory action.
Predictive AI vs. generative AI considerations: Teams have traditionally struggled operationalizing machine learning. A Gartner estimate was that only 50% of predictive models make it to production use cases after experimentation by data scientists. Generative AI, however, offers many advantages over predictive AI, depending on use cases. The time-to-value is incredibly low. Without training or fine-tuning, several functions within different verticals can get value. Today, in seconds, you can generate code (including backend and frontend) for a basic web application. This used to take at least days or several hours for expert developers.
Future opportunities
If you rewound to 2008, you would hear much skepticism about the cloud. Would moving your apps and data from private or public data centers to the cloud ever make sense, thereby losing fine-grained control? However, the development of multi-cloud and DevOps technologies made it possible for enterprises to feel comfortable and accelerate their move to the cloud.
Generative AI today might be comparable to the cloud in 2008. This means that many innovative large companies are still to be founded. This is an enormous opportunity for founders to create impactful products as the entire stack is being built. A simple comparison can be seen below:
Here are some problems that still need to be solved:
Security for AI: Solving the problems of bad actors manipulating models’ weights or making each piece of code have a backdoor written into it. These attacks are so sophisticated that they are easy to miss, even when experts specifically look for them.
LLMOps: Integrating generative AI into daily workflows is still a complex challenge for organizations large and small. The challenge exists regardless of whether you are chaining together open-source or proprietary LLMs. When things break, the questions of orchestration, experimentation, observability, and continuous integration become important. LLMOps tools will be needed to solve these emerging pain points.
AI agents and copilots for everything: An agent is your chef, EA, and website builder. Think of it as an orchestration layer that adds a layer of intelligence on top of LLMs. These systems can let AI out of its box. For a specified goal like: “create a website with a set of resources organized under legal, go-to-market, design templates and hiring that any founder would benefit from,” the agents would break it down into achievable tasks and then coordinate to achieve the objective.
Compliance and AI guardrails: Regulation is coming. It is just a matter of time before lawmakers around the world draft meaningful guardrails around this disruptive new technology. From training to inference to prompting, there will need to be new ways to safeguard sensitive information when using generative AI.
LLMs are already so good that software developers can automatically generate 60-70% of code using coding copilots. This number is only going to increase in the future. One thing to keep in mind, though, is that these models can only produce something that’s a derivative of what has already been done. AI can never replace the creativity and beauty of a human brain, which can think of ideas never felt before. So, the code poets who know how to build unique technology over the weekend will find AI a pleasure to work with and in no way a threat to their careers.
Final thoughts
Generative AI for the enterprise is a phenomenal opportunity for visionary founders to build the FAANG companies of tomorrow. This is still the first innings being played out. Large enterprises, SMBs, and startups are all figuring out how to benefit from this innovative new technology. Like the California gold rush, it might be possible to build successful companies by selling picks and shovels if the perceived barrier to entry is too high.