Can real-time OLAP systems truly support upserts (insert-or-update) without sacrificing performance? In this technical deep dive, learn how Apache Pinot brings native upsert support to real-time analytics — enabling billions of primary keys per node, ingestion freshness SLAs of seconds, and hundreds of thousands of events per second.
Traditional OLAP systems struggle with mutable data and real-time freshness. This talk explores how Pinot’s architecture overcomes those challenges to power modern analytics use cases — from ride-sharing demand prediction to real-time customer profile enrichment.
You’ll learn:
- How Apache Pinot efficiently manages billions of primary keys on a single node
- Techniques for fast recovery and maintaining strict operational SLAs
- Cost optimization strategies as tables scale dramatically
- The design principles behind Pinot’s native upsert implementation
If you’re building real-time analytics on constantly evolving datasets, this session will show you how to achieve both freshness and flexibility — without compromising query speed or cost efficiency.
Deepthi is a Senior Software Engineer at StarTree, where she has extensively worked on Upserts and Deduplication in Apache Pinot, addressing real-time data challenges at scale. She is very passionate about contributing to open-source projects and engaging with Data Engineering community. She holds a Master’s degree in Computer Science from Purdue University.
Recommended resources
Watch more events.
Newsletter
Our strategies and tactics delivered right to your inbox