Dive Brief:
- Accurately forecasting cloud-based data costs is a challenge for most companies, according to a Forrester survey released Thursday commissioned by Capital One. Eight in 10 of 157 data management professionals surveyed cited difficulty predicting data-related cloud costs.
- The same number of respondents pointed to confusing data governance policies and lack of effective data cataloging as cost-management obstacles.
- To rein in data costs and optimize data usage, Capital One federated its data infrastructure management model. The company equipped each of its lines of business with tools and resources to identify inefficiencies and better understand and forecast costs, according to Salim Syed, head of engineering at Capital One Software.
Dive Insight:
Cloud migration promises access to vast amounts of computing power, expansive data storage, and, when not carefully managed, escalating costs.
Spending on data-related services is difficult to predict in the cloud, an environment in which there are few fixed costs. Today’s data piles atop yesterday’s and becomes old data as newer streams are tapped. Storage costs mount.
As modern analytics, visualization and processing applications are deployed and updated, more computing power is needed. The added cost of these services can be overlooked until the bill is due.
“Organizations pay not only for how much data they are using, but also how they are using that data,” Syed said via email. Most cloud platforms charge separately for storage and computing.
To complicate matters, companies are relying on multiple vendors for storage and computing. In the next one to two years, 84% of data management professionals surveyed by Forrester expect their companies to house data with more than one provider. Multivendor use for applications and computing is expected to increase fourfold.
Overall cloud spend is growing, according to two recent Gartner reports. In an April report, Gartner forecast a worldwide public cloud end-user spending increase of over 20% this year, to $494.7 billion, up from $410.9 billion in 2021. Enterprise IT spending on public cloud services is expected to overtake spending on traditional, on-prem computing by 2025, according to a February report by Gartner.
Capital One is invested in data management solutions, both for its needs and as a new line of business. The company developed Slingshot, a Snowflake Data Cloud management system in house and launched Capital One Software in June.
The company also adopted a federated approach to data management, which allows business teams within the company to “move at their own pace with a central team to manage costs and risks,” Syed said.
In addition to taking steps to address inefficiencies in the company’s data storage systems, Capital One uses a “sloped governance approach” that applies different standards and security controls across different types of data based, in part, on potential value and cost.
“Organizations generate value from data in different ways,” Syed said. “Unfortunately, not all use cases generate a profit.”
