Session Overview
Discover how to bridge the gap between fresh, real-time events and years of historical lakehouse data. In this session, Chinmay Soman (VP of Product at StarTree) and Kundan Vyas (Director of Product & Partnerships at StreamNative) break down the architecture needed for full-spectrum analytics.
Learn how combining StreamNative, Apache Iceberg, and StarTree (Apache Pinot) enables sub-second query performance across both live streaming data and petabyte-scale historical archives—without costly duplicate pipelines or complex federation logic.
What You'll Learn
- The core trade-offs between streaming systems and batch data warehouses
- How to detect fraud and security anomalies by comparing real-time login activity against 90-day baselines
- How StreamNative automates ingestion into Apache Iceberg and Databricks Unity Catalog
- How StarTree and Apache Pinot execute unified queries across real-time and external lakehouse tables
- Real-world architecture patterns for cost-efficient, low-latency analytics
Chapters
- 00:00 — Introduction & what is full-spectrum analytics?
- 01:05 — Streaming vs. batch data warehouses: the freshness and retention dilemma
- 02:45 — Key use cases: security, fraud detection, and user behavior
- 04:00 — The scenario: detecting credential stuffing & bot attacks
- 05:35 — Core architectural requirements for unified analytics
- 08:00 — StreamNative overview: Kafka, Apache Pulsar, and Ursa Lakehouse
- 13:10 — Demo: streaming events into Apache Iceberg & Unity Catalog
- 16:05 — StarTree & Apache Pinot architecture overview
- 18:15 — Hybrid tables: combining real-time segments with external lakehouse data
- 20:12 — How query federation and time boundaries work
- 22:15 — Demo: querying hybrid tables & anomalous login dashboard
- 24:35 — Summary and wrap-up
