Streaming data is at the heart of every modern data platform — but building a robust, scalable streaming architecture can be complex. In this talk, we’ll explore how to design and implement efficient streaming pipelines with Apache Iceberg, a next-generation table format that brings reliability and flexibility to data lakes.
You’ll learn how to:
- Identify your organization’s streaming requirements and architecture patterns
- Integrate key tools like Apache Flink, Apache Kafka, Debezium, Kafka Connect, and Apache Spark for end-to-end data movement and transformation
- Manage compaction and delete files in Iceberg to maintain performance and consistency
- Streamline real-time analytics, machine learning, and data lakehouse ingestion with Iceberg
Whether you’re modernizing legacy ETL or scaling up real-time data systems, this session offers practical best practices and design patterns for streaming to Apache Iceberg tables efficiently and reliably.
