A race is on to build bigger and better cloud data centers and AWS is revving its massive infrastructure engine.
In the first half of the year, the largest hyperscaler poured more than $50 billion into its U.S.-based pipeline expansion, pledging $35 billion for multiple sites in Virginia, $11 billion for an Indiana campus and $10 billion for two Mississippi complexes.
Globally, the company laid out plans for new construction in Taiwan, Japan and Singapore and for a 7.8 billion euro ($8.4 billion) European sovereign cloud region in Germany — all announced since January.
Generative AI workloads are fueling the build outs. The company already boasts billions in annual revenue from the technology, Prasad Kalyanaraman, AWS’ VP of infrastructure services, said in a Wednesday blog post. But the company’s infrastructure investments are grounded in broad enterprise hunger for cloud services.
“We're seeing strong demand for both non-generative AI and generative AI workloads and that is driving our need for capacity,” Kevin Miller, AWS VP of Global Data Centers, told CIO Dive. “We build data centers based on the growth of the business. Keeping our data centers highly utilized is a foundational piece of our business.”
Cloud usage is climbing steadily as enterprises leverage hyperscaler infrastructure to modernize operations, shutter on-prem data centers and shift to a service-based IT model. Consumption, in turn, brings revenue. Gartner expects global cloud spending to surpass $675 billion in 2024, a 20% year-over-year increase.
The three largest hyperscalers have reaped revenue rewards and each has plotted out infrastructure expansions. Microsoft, which captured one-quarter of global cloud spend during the first three months of 2024, initiated more than a dozen major data center projects in the last eight months. Google Cloud, the junior player in big cloud with 11% of the market, recently announced billion-dollar build outs in Virginia, Indiana and Missouri.
Despite the competition, AWS has steadily commanded nearly one-third of the growing market. Amazon’s cloud division brought in $25 billion in revenue during the first three months of the year, adding $9.4 billion to the tech giant’s operating income ledger, according the company’s Q1 2024 earnings report.
“The more demand AWS has, the more we have to procure new data centers, power and hardware,” Amazon President and CEO Andy Jassy said during an April earnings call.
“We're at a $100 billion-plus annualized revenue run rate, yet 85% or more of the global IT spend remains on-premises,” Jassy added. “This is before you even calculate GenAI, most of which will be created over the next ten to twenty years from scratch and on the cloud.”
Chips and a dip
AI’s voracious compute appetite is changing data centers from the inside out.
Hunger for graphics processing units and other high-powered chip technologies slowed data center revenue growth during the first half of the year, according to a Dell’Oro Group report. The dip was in large part due to chip supply constraints triggered by the shift to accelerate compute.
It was a temporary slowdown, Lucas Beran, research director, data center infrastructure at Dell’Oro, said.
“Pandemic induced data center buildouts have largely wrapped up but AI deployments have yet to meaningfully start to scale,” Beran said in an email. “Hyperscalers will be the driving force of growth as they have the capital to invest in the CaPex intensive data center build outs related to AI workloads.”
The trend was reflected in a record quarter for chip sales, the analyst firm found. Semiconductor and data center component sales surged 152% year over year during the first three months of 2024 as GPU giant Nvidia saw revenues skyrocket.
AWS isn’t new to the chip game. The company is banking on a diversified portfolio of proprietary silicon and Nvidia GPUs to satisfy growing demand for AI services.
“There is no compression algorithm for experience,” Kalyanaraman said in his post. “Since we’ve been building large scale data centers for more than 15 years and GPU-based servers for more than 12 years, we have a massive existing footprint of AI infrastructure.”
The company plans to have its third-generation Trainium 2 chips ready to install later this year, Miller said. The company will deploy them in massive clusters to accelerate foundation model training.
For more common inferencing operations, which most analysts expect to comprise the bulk of enterprise AI consumption, AWS already has customized Inferentia chips powered up.
“When it comes to inference, there has not been a dramatic change,” Miller said. “It’s a capability that we've had years and years of experience with running and those workloads look similar to any other customer workloads.”
Beyond GPUs
The building boom is an industrywide phenomenon, eating up land, grid power and workforce. AWS is locked in a battle for cloud dominance with Microsoft, Google Cloud and numerous smaller competitors. Nearly every provider is sinking capital into AI data centers.
“These capital programs are competing for scarce resources on the GPU and chipset side,” Brian Alletto, director of enterprise technology at West Monroe, told CIO Dive. “But I think what we’re seeing in the industry is competition for power systems, cooling systems and expertise to build these things out — it's a busy time for that portion of the construction industry.”
The large cloud providers benefit from experienced in-house design and construction teams to guide infrastructure build outs. AWS expects to have its Mississippi data centers ready to spin up capacity as soon as next year, Miller said.
“Hyperscalers certainly benefit from economies of scale with lower cost of infrastructure, space and power,” according to Beran.
But there are myriad considerations and hurdles to overcome to turn the dollars pledged for new construction into operational infrastructure.
“We have to look at grid capacity, land parcels, zoning and working with local governments,” Miller said.
Projects also require workforce resources during and after the construction process.
“Data centers require skilled workforce technical skills to operate the facilities to operate the server equipment,” Miller said. “it comes down to having the right partnerships with local governments, local community colleges and local energy suppliers to make all of this come together.”
