As enterprises move forward on generative AI, CIOs should know that even the latest models available today will quickly become outdated.
By 2028, more than half of enterprises that built custom large language models will abandon initiatives due to costs, complexity and technical debt, according to Gartner research published Wednesday.
“The potential for technical debt is really high for companies that move fast,” Rita Sallam, distinguished VP analyst at Gartner, said. New techniques and models emerge to lower the cost and improve the accuracy nearly daily, she said.
There’s acute technical debt potential for enterprises that have started from square one, but even those that lean on vendor solutions should prioritize being nimble.
As teams build applications with the models of today, CIOs must pick architectures that lend themselves to quick API updates as new ones emerge, Sallam said.
OpenAI pioneered the enterprise generative AI conversation with GPT-3.5, which powered ChatGPT as the tool launched in November 2022. Less than six months later, the startup debuted its next model, GPT-4. By January, OpenAI previewed an updated GPT-4 Turbo model. The company is reportedly working on its next iteration, Q*.
OpenAI isn't the only provider moving quickly to the next model release. Anthropic rolled out its generative AI chatbot Claude in March 2023 with its first-generation model set. The startup is now on its third generation model set with Claude 3, which it released this month.
“Not only are there thousands of models right now, but there are tens of thousands, hundreds of thousands of models coming,” Marc Benioff, CEO at Salesforce, said during the company’s earnings call in February. “All the models that are available today will be obsolete 12 months from now.”
Pros and cons of moving fast
The idea of technical debt might make CIOs recoil, but early efforts to build LLMs aren’t in vain.
Fast-moving enterprises have the upper hand in upskilling initiatives and acquiring skilled talent to support AI aspirations, Sallam said.
“When you move fast, you’re most certainly going to incur more technical debt along the way, but that’s not always a bad thing,” Sallam said. “If your AI ambition is to be a disrupter in your industry, that sort of technical debt can be viewed as a cost of investing in an option that will pay off other strategic dividends.”
While early movers are at risk of technical debt, generative AI efforts may help in that area, too. Gartner predicts that by 2027, enterprises will use generative AI tools to create appropriate replacements for legacy apps, shrinking modernization costs by 70%.
“Your generative AI tech debt will potentially be less of an acute problem than it was in the past, particularly if you’re architecting knowing that there’s going to be stuff you’re going to change,” Sallam said.
The prospect of dwindling tech debt lures enterprises with a wealth of legacy code, even in highly regulated industries. Banks are hoping generative AI can release engineers from the time drain of COBOL.
Vendors are happy to assist on the mission, releasing tools that target legacy code. IBM’s watsonX can translate COBOL into Java, a capability released last summer. Accenture created a tool using GPT-4 and a vector database to help developers with COBOL technical and functional documentation to accelerate modernization.
Balancing the organizations’ AI goals with risk tolerance will guide next steps, Sallam said. Using AI for coding, content generation or productivity via vendor solutions is relatively easy to procure and deploy, offering a lower risk for tech debt for enterprises.
But there’s a catch. “The competitive moat is, if you have any at all, very short-lived,” Sallam said.
