Ryft

StreamNative partners with Ryft to enable real-time data ingestion into Apache Iceberg tables. Ryft provides automatic table compaction, snapshot lifecycle management, and schema evolution for data streamed through Kafka and Pulsar topics — delivering a production-ready lakehouse ingestion layer.

Real-Time Iceberg Ingestion

Ryft consumes streaming data from StreamNative and writes it directly into Iceberg tables with:

• Automatic compaction — small files from real-time ingestion are continuously compacted for optimal query performance
• Snapshot lifecycle management — configurable retention policies for Iceberg snapshots, balancing time-travel capabilities with storage costs
• Schema evolution — seamless handling of schema changes in streaming data without manual table migrations

Lakehouse Architecture

The StreamNative + Ryft combination enables a streaming-first lakehouse architecture where data flows from producers through Kafka/Pulsar topics and lands in queryable Iceberg tables within seconds. Downstream analytics engines like Spark, Trino, and DuckDB can query the latest data without waiting for batch ETL jobs.

Use Cases

Organizations use StreamNative + Ryft for real-time data warehousing, streaming ETL into data lakes, and building low-latency analytics on fresh data without the complexity of managing Iceberg table maintenance operations.