Pulsar Newbie Guide for Kafka Engineers (Part 2): Tenants, Namespaces & Bundles

‍TL;DR

This post demystifies Pulsar’s multi-tenancy model – namely tenants, namespaces, and bundles. Pulsar is built for multi-tenancy from the ground up, unlike Kafka’s single-tenant assumption. A tenant is like a top-level account or project, a namespace is a grouping of topics within a tenant (with its own policies), and bundles are internal shards of a namespace used for load distribution. We’ll explain how these relate to Kafka’s concepts (e.g., Kafka has no direct equivalent, often one Kafka cluster = one tenant). By the end, you’ll understand how Pulsar isolates workloads and balances load across brokers seamlessly using bundles.

Understanding Tenants and Namespaces

Tenants in Pulsar are the highest-level grouping. You can think of a tenant as an account or a logical business unit. For example, if multiple teams or applications share a Pulsar cluster, you might create a tenant for each team (e.g., finance, iot, analytics). This is fundamentally different from Kafka, where usually a whole cluster is managed as one unit (multi-tenancy in Kafka is often achieved by separate clusters or naming conventions). Pulsar was designed with multi-tenancy from day one: “Pulsar was created from the ground up as a multi-tenant system”. Each tenant can be restricted to certain clusters (in geo-replication scenarios) and have its own admin policies like storage quotas or auth rules.

A namespace in Pulsar is a subdivision of a tenant, used to group topics. If a tenant is like a project, namespaces are like environments or categories within that project. For instance, under tenant finance, you might have namespaces transactions, audits, realtime etc. Technically, a namespace is identified by tenant/namespace (for example, finance/transactions). Namespaces are important because they are the unit at which many policies are applied (retention, TTL, anti-affinity, etc.) – “The configuration policies set on a namespace apply to all topics in that namespace”. In Kafka terms, you might compare a namespace to a group of topics that share configs, but Kafka doesn’t have a first-class entity like this – Pulsar’s namespaces are a unique feature to simplify administration.

Every Pulsar topic name includes the tenant and namespace as a prefix. The full name format is:

persistent://tenant/namespace/topicName

For example: persistent://public/default/my-topic. Here public is the tenant (a built-in tenant that comes with Pulsar), and default is the namespace. This structure is why in Part 1 we always included public/default in our topic names. It ensures multi-tenant isolation in the topic naming itself.

Why this matters: In a Kafka cluster, all topics share the same broker namespace and you’d need to implement multi-tenancy via conventions (like topic name prefixes per team) or separate clusters. Pulsar’s approach ensures strong isolation – you can apply permissions at tenant or namespace level, set storage quotas per tenant, and so on, which is much cleaner for multi-tenant use cases.

Creating and Managing Tenants & Namespaces

To create a tenant in Pulsar (which an admin would do), you’d use:

bin/pulsar-admin tenants create my-tenant

This sets up a new tenant. Typically, you will specify which Pulsar clusters this tenant can use (in a multi-cluster deployment) with --allowed-clusters, but in a standalone or single cluster context, defaults are fine. After creating a tenant, you create namespaces under it:

bin/pulsar-admin namespaces create my-tenant/my-namespace

This will create a namespace called my-namespace for my-tenant. By default, a new namespace is created with some number of bundles (more on bundles next) – often 4 bundles by default unless configured otherwise. You’ll notice a public tenant with public/default namespace automatically present for convenience.

You can list tenants with pulsar-admin tenants list and list namespaces in a tenant with pulsar-admin namespaces list tenant (as we saw in Part 1). Also, each namespace can be configured with policies: e.g., message TTL, retention, max consumers, etc., via pulsar-admin namespaces set-policy... commands or in configuration.

What are Bundles?

Pulsar takes the scalability of topics further with namespace bundles. A bundle is essentially a subset of topics in a namespace, defined by a hash range, that can be treated as a unit for broker assignment. When Pulsar needs to balance load, it moves bundles between brokers, rather than individual topics. This is a key difference from Kafka: in Kafka, the unit of load distribution is a partition of a topic. In Pulsar, the unit is a bundle, which may comprise many topics (or just a few) in that namespace. This design allows Pulsar to handle thousands of topics without each topic individually overloading the metadata layer.

Formally, “a namespace bundle is a virtual group of topics that belong to the same namespace”. Each bundle is identified by a range of the 32-bit hash space (0x00000000 to 0xffffffff). By default, a new namespace is created with 4 bundles (which splits that hash range into 4 quarters). You can also specify a different number of bundles at creation time, e.g., --bundles 16 for finer granularity.

Analogy: If a namespace is like a big bucket of topics, bundles break that bucket into (hash) slices. Each slice can be independently moved to a different broker. Kafka doesn’t have an exact analog since Kafka topics are independent; you might compare a Pulsar bundle to a group of partitions from possibly multiple topics that a broker handles. But for simplicity: think of bundles as load-balancing shards of a namespace.

Bundle Load Balancing in Action

Why bundles? Suppose you have 1000 topics in one namespace all on one broker – not ideal if traffic grows. Pulsar’s broker load manager monitors broker load and can move a busy bundle to another broker to spread out load. If one bundle (hash range) of topics is hot (lots of traffic), the broker can unload that bundle: “if the broker gets overloaded with the number of bundles, [you] can unload a bundle from that broker, so it can be served by another broker”. This automatic movement is like Kafka’s partition rebalance, but occurs transparently and can be automatic (depending on load manager settings) – no manual partition reassignment needed for routine balancing.

How it works: Each topic’s name is hashed to determine which bundle it falls into. All topics in the same bundle live on the same broker. If that broker is strained, Pulsar can split the bundle into smaller ranges or move some bundles away:

• Splitting a Bundle: Pulsar supports splitting a bundle that’s too busy into two smaller bundles. This is done by admin command or automatically by some load managers. For example, to split a specific bundle range:
  

bin/pulsar-admin namespaces split-bundle --bundle 0x00000000_0x7fffffff my-tenant/my-namespace

• This would split the bundle covering hashes 0x00000000 to 0x7fffffff into two equal halves. After splitting, one or both of those new bundles could be unloaded to other brokers.
  
• Unloading a Bundle: You can force a bundle off a broker (causing it to be reassigned) with:
  

bin/pulsar-admin namespaces unload --bundle range my-tenant/my-namespace

• For example: ... unload --bundle 0x80000000_0xffffffff my-tenant/my-namespace to unload that specific bundle. The brokers will auto-determine which broker should own it next (often the least loaded one).
  

These operations might sound low-level, but Pulsar can handle much of this automatically. By default, you usually won’t manually split/unload unless debugging or pre-scaling.

From a user perspective, this is mostly transparent. You publish and consume from topics as normal; behind the scenes Pulsar may move the bundle containing your topic to a different broker, but your producers/consumers follow automatically (Pulsar clients get redirected to the new broker). There is no need to manually rebalance as you might in Kafka when adding brokers – Pulsar’s design allows new brokers to immediately take over bundles and thus traffic.

Why Kafka Engineers Should Care

If you ran a Kafka cluster with many topics, you might have encountered the challenge of too many open file handles or uneven load because one topic or partition was hot. Pulsar’s tenants and namespaces encourage you to logically organize topics (so you can apply policies easily), and bundles ensure dynamic load distribution. It’s like having an automatic partition rebalancer always on.

A Kafka engineer might ask: “Can’t we just have one big topic with partitions in Kafka to distribute load?” Yes, but then all those messages are interrelated. Pulsar’s bundles let completely unrelated topics still share brokers efficiently. It’s a different approach to scaling:

• In Kafka, you scale by partitioning a topic and manually balancing partitions across brokers (each partition is stuck to a broker unless you move it).
  
• In Pulsar, you scale by having many topics in a namespace and letting Pulsar move bundles of topics around as needed. You also partition individual topics for parallelism (similar to Kafka), but even those partitions are managed under the hood by bundles and can move.
  

Important note on topic naming and discovery: Because topics include tenant and namespace, tools and APIs will often ask for those. For example, to subscribe or produce you give the full name or use the client API with tenant/ns parameters. If you attempt to use just a topic local name without tenant/ns in Pulsar, it assumes the public/default namespace by default – which is fine for quick tests but not in a structured multi-tenant environment.

Recap and CLI Tips

• You can see all bundles in a namespace via:
  

bin/pulsar-admin namespaces bundles tenant/namespace

• This will list the bundle ranges currently defined.
  
• Default bundle count is 4. If you expect a very large number of topics or high throughput, you might pre-create the namespace with more bundles (e.g., 16 or 32) so load can spread over more brokers from the start.
• ‍Tenants and Namespaces creation requires superuser or tenant admin rights. In a managed environment, tenants would be set up by an admin, namespaces would be set up by either a super user or a tenant admin, and developers might only have access to create topics within an assigned namespace.
• Pulsar isolates data by tenant: one tenant cannot access another’s topics unless explicitly granted permission. This is analogous to completely separate Kafka clusters for different tenants from a security perspective. We’ll touch on permission management more in the Security part.
• The multi-tenancy doesn’t add overhead in usage – it’s mostly an organizational and isolation benefit. Producing/consuming is the same, just with a qualified topic name.
  

Now that we have a handle on how Pulsar’s namespace system differs from Kafka, in the next part, we’ll delve into how Pulsar stores data with BookKeeper – covering Ledgers & Bookies – which is another big architectural difference from Kafka’s log segments on brokers.

Key Takeaways

• Tenants in Pulsar are top-level containers for isolating multiple applications or teams in one cluster. Kafka has no native tenant concept – Pulsar allows true multi-tenancy in a single cluster.
  
• Namespaces are subdivisions of a tenant, grouping topics and applying policies collectively. Think of them as analogous to Kafka topic prefixes or categories, but enforceable and configurable at the broker level.
  
• Bundles are Pulsar’s unit of horizontal scalability within a namespace – a hash range of topics that can move between brokers for load balancing. Kafka’s closest equivalent is partition reassignments, but Pulsar does it proactively and in a grouped manner.
  
• Multi-tenancy and bundles mean adding a new broker to Pulsar can automatically relieve hotspots (brokers share the work by moving bundles), whereas in Kafka you’d manually rebalance partitions or use external tools.
  
• Organizing topics into tenants/namespaces is crucial for Pulsar usage. It might feel unfamiliar to Kafka users, but it provides powerful isolation and flexibility (per-namespace retention settings, encryption policies, etc.). Embrace these concepts to fully leverage Pulsar’s strengths.