In a world where always-on engagement is the norm, availability alone isn’t enough — systems must stay resilient under failure, scale across thousands of tenants, and meet distinct SLAs for every data stream.
At Blueshift, the team rebuilt their data backbone around Apache Pulsar, transforming fragile, tightly coupled pipelines into a resilient, scalable messaging fabric that powers multi-tenant, real-time engagement at a global scale.
In this talk, discover how Blueshift:
- Eliminated cascading failures by replacing synchronous RPC chains with fully asynchronous Pulsar-based messaging.
- Isolated tenants and workloads using thousands of lightweight Pulsar topics, ensuring gold-tier SLAs for critical updates while separating low-priority traffic.
- Streamlined archival with a low-cost, high-throughput pipeline that handles massive data volumes effortlessly.
- Scaled horizontally on demand, adding client-specific capacity during high-traffic periods without downtime.
- Simplified data architecture with multi-subscription fan-out — a single durable topic feeding multiple downstream services without duplication.
💡 Key Learnings:
- Async Communication Excellence: How Pulsar’s asynchronous model reduced latency and decoupled services.
- SLA-Driven Topic Design: Designing topic hierarchies that enforce QoS guarantees efficiently.
- Multi-Tenant Resilience: Managing quotas, throttling, and isolation for predictable performance.
- Archival at Scale: Building efficient streaming-to-storage pipelines.
- Operational Lessons: Real-world tips for monitoring, capacity planning, and troubleshooting Pulsar in production.
If you’re an architect, data engineer, or platform lead building resilient and scalable data infrastructure, this session delivers concrete architectural patterns and hard-won lessons from Pulsar in production at enterprise scale.
