Querying large volumes of data from a data lake demands optimized query speed.

Apache Hudi offers efficient data management capabilities such as “clustering” and “metadata table” that provide significant query performance improvements for compute engines like Presto.

Traditionally, during the ingestion process, data is organized according to its arrival time. Yet, for optimal performance, query engines require that frequently accessed data be co-located together.

Hudi's clustering service significantly reduces query latencies by automatically file sizing and laying out data optimally for engines like Presto, thereby enabling faster data retrieval and processing.

Additionally, Hudi's metadata table eliminates the performance bottleneck of extensive file listing operations in cloud object stores like AWS S3 by proactively maintaining the list of files, ensuring a more efficient and optimized data querying process.

In this talk, we will demonstrate how leveraging these Hudi capabilities with Presto enables users to achieve unparalleled query speeds, ideal for interactive ad hoc analytics.