Session Overview
When AI agents rely on a data lakehouse for long-term memory, subtle pipeline bugs can silently feed them out-of-order, stale data. Because the agent doesn't crash, it gives wrong answers with total confidence.
In this talk, Spoorthi Basu (Software Engineer at Genesys) explains how out-of-order commits and routine table compactions silently overwrite valid data. Learn how Flink CDC and Apache Iceberg equality deletes interact, why standard "exactly-once" semantics won’t protect against read-time data anomalies, and how to detect stale winner rows before compaction washes the evidence away.
What You'll Learn
- How out-of-order CDC updates cause stale records to win over fresh data
- The checkpoint schema-change bug in Flink CDC and how it was resolved
- JobManager off-heap memory exhaustion during high-split source snapshots
- Why table compaction acts as "evidence laundering" for a corrupted state
- Retention clock tradeoffs across Iceberg, Apache Hudi, and Delta Lake
- How to use a read-only checker and metadata gating to identify and backfill stale keys
Chapters
- 00:00 — Introduction & the silent agent memory problem
- 02:46 — Flink CDC schema bug & equality deletes
- 06:22 — Processing guarantees vs. read-time correctness
- 07:03 — JobManager memory leaks in high-split snapshots
- 11:15 — Detecting stale winners with a read-only checker
- 13:26 — How compaction hides out-of-order data corruption
- 15:45 — Retention clocks across Iceberg, Hudi, and Delta Lake
- 17:26 — Safe pre-combine configurations & best practices
- 19:31 — Compaction patching & targeted backfills
- 25:44 — Conclusion & wrap-up
