Dive Brief:

• Big data software companies Cloudera and Hortonworks announced a merger agreement on Wednesday under which the latter will become a subsidiary of Cloudera. Upon close of the merger, the new Cloudera board of directors will consist of four Hortonworks directors and five Cloudera directors. Current Cloudera CEO Tom Reilly will retain his position at the head of the reorganized companies and also serve on the board, according to the Cloudera 8-K filing.
• The merger will combine the complimentary companies' portfolio into an enterprise data hybrid cloud platform, Reilly said on conference call Wednesday. He stressed that the merger plays off both companies' strengths, tying together Hortonworks' internet of things technology at the edge focus with Cloudera's investments in data warehousing, artificial intelligence and machine learning.
• The companies will immediately have more than 2,500 customers, many large ones of which are not in common, opening up sales opportunity for the companies' unified "Edge to AI" platform, Bearden said on the call. The companies' will support the distinct Hortonworks and Cloudera platforms for at least three years for existing customers, Reilly said. After the merger, a unity release of a new core platform will be available for customers to join.

Dive Insight:

For companies that rose to prominence building Hadoop frameworks for customers, the cloud has posed an existential threat. Customers expect to "experience the cloud inside their own data centers just like they do in the public cloud," necessitating a public cloud experience across architectures for widespread adoption, Reilly said during the conference call. 

Hadoop rose to prominence for batch workloads, but in the era of real-time data and event-driven applications, the technology isn't always fitting to the space.

The Hadoop premise was taking data from everywhere and making big data, but this left many businesses with piles of data they weren't doing anything useful with, said Yaron Haviv, founder and CTO of Iguazio, in an interview with CIO Dive. 

Now, data is leveraged across business applications in the short term to generate business value. The value of data diminishes over time, so businesses need to be able to act on it quickly. This is where cloud capabilities and advanced computing techniques come in. 

But a less rosy view of the merger would take into account the declining prevalence of the Hadoop framework, an open-source software for data storage, orchestration and computing. It's a 10-year-old technology, and many of its layers are being cannibalized by the cloud, Haviv said. 

At the storage layer, more customers are moving their data to the cloud; offerings such as Amazon Web Services S3 are cheaper, more scalable and fully provisioned, he said. And at the orchestration layer, frameworks such as Kubernetes are more general purpose than Hadoop. Finally, at the computation layer, many cloud providers are also providing functions such as AI and ML to help customers manipulate their data. 

The consolidation of these two players will create a more obvious market leader that can pursue similar customers without the worry of competition from a similar peer. 

While executives touted the complimentary portfolios of the two companies coming together under a newly unified platform, Haviv noted there was already huge overlap in the companies' portfolios. Hortwonworks being more focused in the IoT space was less a 180-degree departure from Cloudera as a 15-degree one, he said.