In the two years since OpenAI’s ChatGPT made its public debut, vendors rushed to capitalize on the momentum, releasing hundreds of generative AI capabilities and promising transformative benefits.
But CIOs aren’t as starry-eyed as they initially were when the AI race kicked off.
ChatGPT’s ease of use and novelty lured businesses that sought quick productivity gains. Yet, leaders found deploying generative AI at scale is much harder and more complex than the minimalist chatbot veneer suggested.
“We’ve got a problem, generally in the industry, where people equate tools for productivity,” said Tony Marron, managing director of Liberty IT, a subsidiary of Liberty Mutual. “There’s a big difference between putting a tool in people’s hands and giving them the skills to use it.”
If 2023 was the year of generative AI experimentation and crafting a plan, 2024 saw enterprises ramp up execution, and it didn’t go as intended. CIO skepticism of vendors increased alongside the weight of failed initiatives and unrealized gains, leading to more caution toward the technology and an emphasis on guardrails to strengthen governance.
Adoption roadblocks are piled high for less modernized organizations.
The technical maturity needed to scale projects has pushed organizations to accelerate cloud migrations, update data management, beef up security and launch widescale upskilling initiatives. Despite the progress, just 2% of organizations have the talent, data and technology to succeed, according to an Infosys survey published last month.
We’ve got a problem, generally in the industry, where people equate tools for productivity.
Tony Marron
Managing director of Liberty IT
Organizations farther along in their modernization journey haven’t yet cracked the code either, but leaders at those organizations are often more hopeful.
“It has gotten more polarized,” Amanda Luther, managing director and partner at Boston Consulting Group, told CIO Dive. Excitement has increased in companies where generative AI has produced value, but the average organization is “more skeptical,” Luther said, speaking of her conversations with enterprise leaders.
CIOs are under immense pressure from all sides, Luther said. Businesses have allocated a sizable amount of resources and effort to embracing generative AI, underlining the need to deliver results. Initiatives typically touch every business unit, placing more eyes and scrutiny on technology leaders.
“I hear a lot of, ‘We’ve been talking about this for two years, who’s actually seeing anything from this? We’ve invested a lot of dollars and have nothing to show for it,’” Luther said. “That tone of conversation has increased over the last six months.”
Ballooning costs tied to generative AI are a major concern for technology leaders. When ChatGPT entered the enterprise lexicon, people primarily accessed the tool for free online. Now, the cost of add-on generative AI services, built-in capabilities or customizing tools is adding up.
The cost of OpenAI’s ChatGPT Enterprise plan, released in August 2023, varies by the number of seats, extra capabilities needed and scope of deployment, as does Anthropic’s Claude Enterprise, which launched in September. Microsoft 365 Copilot costs around $360 per user yearly and requires a separate qualifying Microsoft 365 plan. It’s another expense enterprises have to manage and monitor, or risk spiraling out of control.
AI workloads have also pushed up cloud costs, which rose an average of 30% in the last year, according to Tangoe research. IT leaders blame AI for increasing software costs, too.
On top of that, many generative AI projects have failed or never made it to production.
“There’s an opportunity for us to be much more intentional about what are the problems that we really want to solve with AI,” Christina Janzer, Slack SVP, research and analytics, said. “Let’s not just introduce AI, for the sake of introducing it.”
Prioritizing and customizing
Enterprises want the efficiency and productivity generative AI vendors promised. At the same time, CIOs can’t chase every use case.
Countless organizations, including General Mills, are working to find that balance, according to Jaime Montemayor, chief digital and technology officer at General Mills.
Shortly after ChatGPT launched, General Mills set up a cross-functional senior leadership team to craft a vision for generative AI at the company, leaning on prior modernization efforts. The food industry giant rolled out an internal generative AI tool called MillsChat in February. Around 20,000 employees across the company use the tool, built using Google’s PaLM 2 model, for writing, summarizing and brainstorming.
“We’re in a very good place as a leadership team,” Montemayor said during a CIO Dive and CFO Dive live event in October. “We made sure that the board was fully aligned with our posture on generative AI and deploying generative AI comes with some risk.”
Montemayor said he provides a quarterly update to the senior leadership team on the overall success of AI initiatives, sharing the good and the bad, so everyone is on the same page and the enterprise can pivot if necessary.
“We’re very proactive in managing people’s expectations, but more importantly, ensuring that we’re delivering value,” Montemayor said. “We are focused on a handful of areas where we know there is value to be generated.”
Businesses are actively working to narrow their list of use cases to pursue as early adopters struggle with ROI and success metrics. Leading organizations tackle about half as many AI opportunities on average compared to less advanced businesses, according to BCG data published in October.
Some companies learned the hard way that generative AI is not the best tool for every occasion. One-quarter of IT leaders regretted investing in AI so quickly, according to an Asana survey published in March.
“Generative AI has many holes,” Rita Sallam, distinguished VP analyst and Gartner Fellow in the data and analytics team, said during Gartner’s IT Symposium/Xpo in October. “It’s strong in some areas, but weak in others and it’s hard to know in advance which is which.”
As the hype faded for IT leaders through trial-and-error, experimentation and best practice gathering over the past two years since ChatGPT’s public introduction, there’s also been a shift away from general-purpose models.
ChatGPT, and a swarm of similar tools that followed, offered enterprises a broad knowledge base that was sometimes wrong. Nearly 9 in 10 technology leaders said it may not be possible to know if AI outputs are accurate, according to a Juniper Networks survey published in February. Industry-tailored tools and customized models can give organizations more control over training datasets and access management, smoothing existing barriers related to hallucinations and accuracy.
“When you get to these really specific tasks that an enterprise needs to solve on their data, on their tool, on their products, these systems don’t just work out of the box,” CrowdStrike CTO Elia Zaitsev told CIO Dive. “We’re starting to see it transition from largely a consumer novelty into a genuine problem-solver for the enterprise.”
CrowdStrike has several generative AI initiatives and solutions underway. One project has focused on combining commercial models with open-source and internal data.
“We’re not just giving it a little bit of additional context here and there,” Zaitsev said. “We’re actually jamming it with a ton of cybersecurity and specific information to make these models even more advanced, even more purpose-built for our use cases.”
Looking forward
As enterprise strategies have matured over the past two years, CIOs have worked to distill priority use cases and implement tailored tools all in an effort to propel the business closer to its goals.
Businesses cast a wide net in 2023, BCG's Luther said. After experimenting widely and investing in ChatGPT, in 2024 enterprises found that, “Wait a minute, none of this is showing up in my P&L. I can’t go talk about this in my quarterly earnings release."
Despite the strategy and sentiment shifts, some things might never change, or at least haven’t yet in the two years since ChatGPT’s launch. CIOs can expect vendors to keep pushing generative AI-powered tools and services. They’ve promised as much.
Enterprisewide generative AI initiatives will also continue to be challenging.
From talent to processes, businesses face a tangled web of organizational and technical debt. The speed of innovation has caught less modernized organizations off guard. More than half of AI decision-makers worry about IT teams’ ability to keep pace, according to a Hitachi Vantara report.
The pressure to deliver is unrelenting for CIOs, but many are determined to take on the challenge.
We’re starting to see it transition from largely a consumer novelty into a genuine problem-solver for the enterprise.
Elia Zaitsev
CTO at CrowdStrike
Kroger is using generative AI across the business, according to Todd James, chief data and technology officer at Kroger-owned subsidiary 84.51°, leading enterprise data and AI at the grocer.
After creating a strategy that aligned with business priorities, the grocer worked to gain employee buy-in this year through in-person and multimodal instruction.
“We have presence across supply chain, merchandise and marketing, and increasingly in manufacturing,” James told CIO Dive. The majority of employees are using AI one way or another, he said.
The strategy for 2025, James said, is simple: get more sophisticated, from managing projects and costs to the actual use cases and democratizing access.
“I want to get AI in more people’s hands, and I want more people developing, but I want it done in an orchestrated fashion,” James said. “That’s what we’re focused on aggressively for next year because it's a key enabler of increasing value for our customers and for our associates.”
