Overview
At Flipkart, there are multiple use-cases for high throughput messaging like streaming/batch pipelines, ordered processing, auditing, etc. Most of the teams manage and deploy their own messaging backends which are primarily Kafka clusters. We identified that offering topic-as-a-service can take away operational complexity for these teams and help us enforce stricter SLAs around uptime and geo-replication. This session will talk about our approach towards building a scalable and multi-tenant platform with Pulsar as the choice of backend.
We maintain our own control plane which offers simpler contracts for onboarding a new user rather than bogging them down with cluster management aspects. The talk will cover some of the design decisions that have been made when offering this as a service in our private cloud and also some of the technical aspects.
The talk will briefly touch upon the challenges we face when driving adoption and convincing kafka users to move to this platform.
Download the slides.
