Before diving into an LLM-based startup, you should think through these five questions carefully. Failing to do so is a recipe for trouble down the road.
Since ChatGPT launched, two trends have dominated the startup and investment world — and both are misguided. First, a flood of startups plan to use large language models (ChatGPT or GPT-4) to provide better solutions to known problems. This is like taking a pre-baked cake and adding frosting, hoping it becomes gourmet. Second, these startups are baking so many of these cakes that investors have started fixating on the frosting itself — they now emphasize the depth of the technology layer on top of the LLM. Many recent successful fundraises have been pitching frosting with different colors, flavors, or textures.
Unfortunately, that’s far from enough. To be clear, this isn’t saying a team can’t overcome these issues — it’s saying that unless they do, they’re likely heading for trouble.
LLM Output Is Unpredictable
Generative AI is notoriously unpredictable. OpenAI provides a temperature parameter to control model creativity. Even set to zero, even with the new function calling feature, the model still produces non-deterministic results. Just try asking ChatGPT the same question repeatedly and you’ll see this firsthand.
For humans, this variability is fine — it even makes the bot feel more human. But for computer systems, this variability is hard to manage. It can stem from subtle prompt variations, edge cases in user messages, or seemingly no reason at all.
While LLMs provide immense power for solving problems, teams often spend more time adapting to the LLM’s input/output format than actually building the downstream application. Multiple teams have built their first fully LLM-based solution within a week. Then, as they try to scale up and deal with unmanageable unpredictability, they start reverting to more traditional, structured approaches. Each such reversal strips the solution of the power the LLM promised.
LLM Providers Will Build the Applications Themselves
Since launching ChatGPT on November 30, 2022, and subsequently GPT-4, OpenAI has added multiple features to its API: plugins, JSON building, function calling, and more. On the surface, “this makes the LLM more predictable.” But you’re missing the point —
OpenAI’s mission is to make LLMs accessible to everyone. That’s how they plan to develop AGI. To that end, they constantly release new versions of GPT and continuously add features to each version. In just months since ChatGPT/GPT-4 launched, OpenAI has added plugins, function calling, better “steering” controls, and Whisper model integration.
Each addition and enhancement makes your innovations less necessary. In fact, I’d bet that most successful things you or other startups innovate will eventually be added to GPT in some form. They’re not being malicious — they’re just following their mission. The same goes for other LLM providers. In other words, once you succeed, the LLM will replicate it so the next person — including your customers — can easily do what you’re doing.
LLM-Based Solutions May Not Be Worth It
LLMs are an amazing invention that will change the pace of technological innovation. However, they face countless issues with predictability, hallucinations, and more. If you’re applying LLMs as a new way to solve existing problems without reimagining the problems themselves, you haven’t thought it through.
Even with an LLM-based solution, the competition is fierce. You’re not only competing against other LLM solutions — and the barrier to entry is dropping fast — you’re also competing against OpenAI itself and every existing solution to that problem.
We should also acknowledge that current solutions may not solve the problem perfectly. But they may already do a good enough job at low cost. If it works well, there’s no reason to disrupt it. If it doesn’t work well, it has already created a massive expectation barrier in customers’ minds that your marketing or sales cycle must overcome.
So unless you’re redefining the problem or creating an experience with no reference point, pause your idea. Even OpenAI went through GPT-1, GPT-2, GPT-3, and then GPT-3.5 before solving some real industry problems. Many other companies simply couldn’t convince the market or investors.
LLMs Look Like a Shortcut, But They’re Not
Despite GPT changing everything, practically nothing has changed. In summary: even with LLMs, building a successful business still requires the same investment in product, engineering, and scientific innovation as before. This is necessary for workflow integration, user experience, control, cost, advocacy, customer branding, and more.
When you invest at that magnitude, what you’re building is no longer an LLM-based solution. Rather, it’s a solution that relies on innovations in product, engineering, and science. In other words, the LLM is just a companion on your journey, not the destination. You still need to work just as hard as before LLMs to create sustainable competitive advantages.
LLMs Have Serious Security Issues
Last but by no means least is enterprise security. We’ve already discussed control issues with generative AI, which is bad enough. Another critical issue is that everything an LLM processes needs to be transmitted to the cloud, which is unacceptable for most enterprises.
LLM providers like OpenAI and Anthropic have reasonable and transparent security policies. However, it’s not 100% reliable — they can’t guarantee they won’t look at the data themselves. This is the age-old information security maxim: once it leaves your firewall, you really can’t control it.
Another option some startups consider is hosting proprietary LLMs within their (or their customers’) firewall. This is a reasonable choice, but now the LLM is no longer an easy-to-use tool. On top of innovating in product, engineering, and domain-specific science, you also have to innovate on the LLM itself.
Again, this post isn’t meant to discourage LLM entrepreneurship or LLM projects. It’s simply a reminder — some things worth thinking deeply about before you start planning.