Session Overview
Can you scale message throughput without sacrificing strict key ordering? In this session, we dive into how Apache Pulsar 5.0 handles ordered data streams at scale, eliminating common trade-offs between partitioning, performance, and message sequencing.
What You'll Learn
- The core challenges of preserving key order in distributed messaging systems
- How Apache Pulsar 5.0 redesigns topic scalability and partitioning
- Practical strategies to optimize stream performance while maintaining deterministic ordering
- Best practices and architectural patterns for high-throughput enterprise workloads
Chapters
- 00:00 — Introduction & speaker background
- 00:33 — Why ordered processing matters & key challenges
- 02:01 — The partition-based dilemma: Ordering vs. scaling
- 02:41 — Pulsar's scalable architecture background
- 04:06 — Key_shared subscriptions & hash routing limits
- 06:04 — Hot shards & head-of-line blocking
- 07:20 — Segmented key space approach: Splitting & merging
- 08:26 — Routing & keeping event order
- 10:30 — Industry approaches & scalable topics in Pulsar 5.0
- 13:32 — Java client API updates (Stream, Queue & Checkpoint)
- 15:32 — Handling hot shards & hot keys with entry buckets
- 18:49 — Splitting triggers & future broker optimizations
- 20:09 — Acknowledgements in Pulsar 5.0 & consumer-side parallelism
- 22:45 — Summary: Scalable ordering for AI & real-time workloads
