From Data to Intelligence — Cost, Lakehouse, and AI Agents | Data Streaming Summit 2025 Keynote
How do we bridge streaming, analytics, and AI into a single, intelligent system that scales without runaway cost? At Data Streaming Summit 2025, StreamNative presented a bold new blueprint for the Agentic Era — a time when real-time data, lakehouse architectures, and AI agents converge into one unified, incremental stack.
This keynote session brings together leaders from StreamNative, Databricks, LinkedIn, and OpenAI to explore the technologies, architectures, and success stories shaping the future of intelligent data infrastructure.
🔷 A Unified Blueprint for the Agentic Era
The keynote opens with a connected story that doubles as a roadmap for the next decade of data infrastructure: Stream the data. Accelerate the insights. Empower the agents. From ingestion to decision-making, this architecture bridges motion (streaming), rest (lakehouse), and action (AI agents) — redefining how real-time intelligence is built and delivered.
⚙️ Ursa — The Lakehouse-Native Streaming Engine
StreamNative CTO Matteo Merli unveils Ursa, a new lakehouse-native engine built to bend the cloud cost curve. Ursa unifies low-latency streaming with cost-optimized object storage, eliminating connector overhead, cross-AZ replication costs, and complex synchronization layers. Now production-ready as a Pulsar storage extension, Ursa enables enterprise workloads to scale efficiently — achieving high throughput, low latency, and significant cost savings.
🧩 Unity Catalog + Iceberg — Streams as First-Class Tables
Kundan (StreamNative) and Michelle (Databricks) reveal how the Unity Catalog and Apache Iceberg integration transforms streaming topics into instantly queryable, governed tables. This native integration allows organizations to unify governance, enforce fine-grained access control, and eliminate data duplication — effectively bridging the gap between streaming and the lakehouse.
🤖 Orca — Bringing AI Agents into the Event Fabric
Neng Lu, Director of Platform Engineering at StreamNative, introduces Orca, a streaming-native runtime for AI agents. Python-first and framework-agnostic, Orca allows agents to be deployed and coordinated directly inside event streams, bringing state, governance, replay, and dynamic tool discovery to real-time AI systems. A live demo showcases how agents scale, recover, and collaborate autonomously — turning event streams into an intelligent, self-evolving fabric.
🏗️ Architectures That Validate the Vision — LinkedIn & OpenAI
Two global leaders shared how they are redefining large-scale streaming architectures:
- LinkedIn unveiled NorthGuard, a log store handling 32 trillion records per day, with elasticity, self-balancing clusters, and segment-based replication.
- OpenAI demonstrated how real-time streaming powers model training, experimentation, and conversational AI, achieving 80 GB/s throughput and 3× quarterly growth using Kafka, Flink, and custom infrastructure.
🚗 Proof in the Wild — Motorq’s Transformation Story
Motorq, a connected-vehicle intelligence platform, showcased tangible results after adopting StreamNative and Ursa:
- 50% lower streaming costs
- Faster lakehouse ingestion
- Near real-time analytics with simplified pipelines and reduced sync overheadThis proves how the new architecture delivers measurable efficiency and performance at scale.
🔥 Fireside Chat — Where the Lakehouse Goes Next
The keynote closes with an inspiring conversation between Sijie (StreamNative CEO) and Reynold Xin (Databricks Co-founder & Chief Architect). They discuss how governance, open formats, and single-file commit semantics will shape the future of the lakehouse — and why streaming-native architectures are essential for reliable, large-scale AI and agentic systems.
🎯 Takeaways:
- Understand how real-time streaming, lakehouse, and AI agents converge
- Learn cost-optimization strategies for large-scale cloud systems
- Explore how leading organizations build next-generation data infrastructure
- Gain insight into open standards and design principles for the Agentic Era
Whether you’re a data architect, AI engineer, or system designer, this keynote provides a comprehensive, forward-looking view of how data becomes intelligence — from motion to action, and from cost to insight.
