You can install and run StreamNative Platform on on-premises environments with a ZIP/TAR archive, or a DEB/RPM package.
You can also automate deployment to Kubernetes on a public or private cloud using Helm.
This document outlines the general guidelines of deploying a production-ready multi-node StreamNative Platform cluster. It applies to both On-Premises deployments and Kubernetes deployments.
StreamNative Platform deployment is mainly comprised of two parts:
The first part is an Pulsar cluster, which includes all the Pulsar open-source components, and StreamNative community and commercial components.
The second part is StreamNative Control Center, which uses Prometheus for collecting metrics, Grafana for visualizing metrics in dashboards, Alertmanager for handling alerts, and Pulsar Manager for managing and operating Pulsar clusters and resources.
The figure below outlines a typical deployment of a StreamNative Platform deployment. The StreamNative Control Center does not have any tight dependencies on an Apache Pulsar cluster. A StreamNative Control Center can be used for monitoring, managing, and operating multiple Apache Pulsar clusters.
Pulsar clients use a
ServiceUrl to connect to an Apache Pulsar cluster to produce and consume messages, as well as manage and operate Pulsar resources (such as tenants, namespace, topics, etc).
So before deploying an Apache Pulsar cluster, you need to plan how to expose the Pulsar cluster to your applications for their usage. There are two approaches:
If you expose brokers directly, the Pulsar applications should run in the same network environment with your brokers. Because Pulsar clients are required to establish TCP connections directly with each of the brokers.
The figure below illustrates a typical setup of an Pulsar cluster exposing brokers directly to Pulsar applications.
If you plan to run a Pulsar cluster in a network environment (such as Kubernetes) where the Pulsar clients can not establish TCP connections directly with each of the brokers. You need to set up a set of Pulsar proxies in front of the brokers. So the Pulsar proxies can look up the ownerships of topics and establish the direct TCP connections with the right owner brokers.
The figure below illustrates a typical setup of an Pulsar cluster exposing the service to Pulsar applications using Proxy.
In both cases, you can configure a DNS and a load balancer for your brokers or proxies. So the Pulsar applications can use one single domain to connect to the cluster. If it is non-trivial for you to set up a DNS or a load balancer, you can configure your Pulsar applications to access the cluster using a
Multi-Host service URL comprised of a list of bootstrap brokers (or proxies) (for example,
There is a certain sequence when deploying a Pulsar cluster. The sequence is highlighted with red numbers in the above figures and described as below.
bin/pulsar initialize-cluster-metadata. The tool initializes the metadata for both Pulsar and BookKeeper and creates a cluster configuration with its service URLs.