Cloudera upgrades platform with Iceberg REST & AI cost savings

Cloudera has announced new updates to its data platform, during EVOLVE25 in New York, highlighting enhancements to Cloudera Iceberg REST Catalog and Cloudera Lakehouse Optimizer.

The latest updates are focused on supporting open, unified data lakehouse environments by addressing enterprise challenges such as complex data architectures, disparate platforms, and the need for consistent governance across multiple environments.

These features are aimed at enabling secure, governed data access, improved performance, and cost savings for organisations seeking to accelerate their artificial intelligence (AI) and analytics initiatives.

Iceberg REST Catalog expansion

Cloudera is integrating the Iceberg REST Catalog into its platform, allowing third-party engines direct access to data managed with Cloudera without the need to copy or move it.

This means organisations can now make use of zero-copy data sharing, which helps to lower costs, reduce security risks, and maintain consistent access policies and metadata intelligence across cloud, data centre, and edge deployments.

Cloudera positions itself as the only vendor in the market delivering unified security, governance and interoperability throughout the complete data lifecycle.

This extends from real-time data ingestion to large-scale processing and ultimately to AI and business intelligence consumption.

The company indicates that by extending Apache Iceberg with REST-based access, enterprises can build more future-proof data strategies, achieve better control and visibility, and meet compliance requirements more effectively.

Following these enhancements, all Cloudera customers using Iceberg can benefit from interoperability with a variety of analytics and AI engines, including Snowflake, Databricks, AWS Athena, AWS EMR, and Salesforce.

The offering promises full ACID compliance and unified access policy enforcement, including fine-grained access controls, data lineage, and auditing, even when using third-party tools via Cloudera's Shared Data Experience (SDX).

The new capabilities also grant open metadata access, which makes data assets discoverable without locking customers into proprietary catalogs.

This change is expected to speed up AI development and business intelligence activities by providing a consistent and accessible source of data truth. Customers have reported notable cost savings, with some indicating up to a 79% reduction in data storage expenses and enhanced visibility across business units. As an example, a global satellite services provider achieved these savings while reinforcing its AI data pipelines.

Lakehouse Optimizer automation

The Cloudera Lakehouse Optimizer is introduced as a service delivering automated optimizations and table maintenance for Apache Iceberg tables.

The company claims that this goes beyond basic housekeeping tasks by supporting actions such as rewriting manifest and position delete files, which are critical for sustaining efficient query performance and storage usage over time.

This optimiser's automation is designed to reduce or eliminate manual data management and operational overhead, freeing up customers to focus on value-generating analytical activities.

It features enterprise-grade observability through a user interface that allows for granular policy setting and adjustment at both table- and catalog-levels. The solution is positioned as open and compatible with any Iceberg-compliant engine across public clouds, with the ability to run on-premises in a future release.

Internal Cloudera benchmarks show potential improvements in query performance by up to 13 times and storage cost reductions by 36% after implementing the Lakehouse Optimizer.

"Cloudera pioneered the Big Data industry, and remains a leading platform provider who continues to invest in making the Apache Iceberg open table format enterprise ready," said Leo Brunnick, Chief Product Officer, Cloudera.

"With today's news, we continue to deliver on the promise of flexibility, scalability, and uncompromised insights when and where you need them the most. That commitment is why the world's largest organizations rely on Cloudera to bring AI to their data-wherever it lives."

Both the Cloudera Data Sharing capability powered by the Iceberg REST Catalog and the Cloudera Lakehouse Optimizer are now available for customers seeking greater flexibility and control over their data architecture.