Announcing: The Pulsar PMC Published The 2020 Apache Pulsar User Survey Report!

Overview
Get started
Install and upgrade
Configure
Overview
Pulsar core
ZooKeeper
BookKeeper
Broker
KoP
Tiered storage
Pulsar Functions
WebSocket
Proxy
Control center
Tool
Secure
Manage and monitor
Connect
Process
Release notes

Configure Pulsar Proxy

Pulsar proxy is a component of StreamNative Platform. It is an optional gateway that you can run in front of the brokers in a Pulsar cluster. You can run a Pulsar proxy in cases when direction connections between clients and Pulsar brokers are either infeasible, undesirable, or both, for example when you run Pulsar in a cloud environment or on Kubernetes or an analogous platform.

Configure the proxy

The proxy must have some way to find the addresses of the brokers of the cluster. You can do this by either configuring the proxy to connect directly to service discovery or by specifying a broker URL in the configuration.

Option 1: Use service discovery

Pulsar uses ZooKeeper for service discovery. To connect the proxy to ZooKeeper, specify the following in conf/proxy.conf.

zookeeperServers=zk-0,zk-1,zk-2
configurationStoreServers=zk-0:2184,zk-remote:2184

If you use service discovery, the network ACL must allow the proxy to talk to the ZooKeeper nodes on the zookeeper client port, which is usually 2181, and on the configuration store client port, which is 2184 by default. Opening the network ACLs means that if someone compromises a proxy, they have full access to ZooKeeper. For this reason, using broker URLs to configure the proxy is more secure.

Option 2: Use broker URLs

The more secure method of configuring the proxy is to specify a URL to connect to the brokers.

Authorization at the proxy requires access to ZooKeeper, so if you use these broker URLs to connect to the brokers, you should disable the Proxy level authorization. Brokers still authorize requests after the proxy forwards them.

You can configure the broker URLs in conf/proxy.conf as follows.

brokerServiceURL=pulsar://brokers.example.com:6650
brokerWebServiceURL=http://brokers.example.com:8080
functionWorkerWebServiceURL=http://function-workers.example.com:8080

Or if you use TLS:

brokerServiceURLTLS=pulsar+ssl://brokers.example.com:6651
brokerWebServiceURLTLS=https://brokers.example.com:8443
functionWorkerWebServiceURL=https://function-workers.example.com:8443

The hostname in the URLs provided should be a DNS entry which points to multiple brokers or a Virtual IP which is backed by multiple broker IP addresses so that the proxy does not lose connectivity to the pulsar cluster if a single broker becomes unavailable.

The ports to connect to the brokers (6650 and 8080, or in the case of TLS, 6651 and 8443) should be open in the network ACLs.

Note that if you do not use functions, then you do not need to configure functionWorkerWebServiceURL.

If you want to change the configuration, you can edit the ${PLATFORM_HOME}/etc/pulsar/proxy.conf file.

The configurations are listed as below:

Name Description Default
zookeeperServers The ZooKeeper quorum connection string (as a comma-separated list)
configurationStoreServers Connection string of configuration store (as a comma-separated list)
zookeeperSessionTimeoutMs ZooKeeper session timeout (in milliseconds) 30000
servicePort Port used to serve binary Protobuf requests 6650
servicePortTls Port used to serve binary Protobuf TLS requests 6651
statusFilePath Path for the file used to determine the rotation status for the proxy instance when responding to service discovery health checks
authenticationEnabled Whether authentication is enabled for the Pulsar proxy false
authenticateMetricsEndpoint Whether the '/metrics' endpoint requires authentication. Default value is true. 'authenticationEnabled' must also be set for this to take effect. true
authenticationProviders Authentication provider name list (a comma-separated list of class names)
authorizationEnabled Whether authorization is enforced by the Pulsar proxy false
authorizationProvider Authorization provider as a fully qualified class name org.apache.pulsar.broker.authorization.PulsarAuthorizationProvider
brokerClientAuthenticationPlugin The authentication plugin used by the Pulsar proxy to authenticate with Pulsar brokers
brokerClientAuthenticationParameters The authentication parameters used by the Pulsar proxy to authenticate with Pulsar brokers
brokerClientTrustCertsFilePath The path to trusted certificates used by the Pulsar proxy to authenticate with Pulsar brokers
superUserRoles Role names that are treated as “super-users,” meaning that they will be able to perform all admin
forwardAuthorizationCredentials Whether client authorization credentials are forwared to the broker for re-authorization. Authentication must be enabled via authenticationEnabled=true for this to take effect. false
maxConcurrentInboundConnections Max concurrent inbound connections. The proxy will reject requests beyond that. 10000
maxConcurrentLookupRequests Max concurrent outbound connections. The proxy will error out requests beyond that. 50000
tlsEnabledInProxy Whether TLS is enabled for the proxy false
tlsEnabledWithBroker Whether TLS is enabled when communicating with Pulsar brokers false
tlsCertificateFilePath Path for the TLS certificate file
tlsKeyFilePath Path for the TLS private key file
tlsTrustCertsFilePath Path for the trusted TLS certificate pem file
tlsHostnameVerificationEnabled Whether the hostname is validated when the proxy creates a TLS connection with brokers false
tlsRequireTrustedClientCertOnConnect Whether client certificates are required for TLS. Connections are rejected if the client certificate isn’t trusted. false
tlsProtocols Specify the tls protocols the broker will use to negotiate during TLS Handshake. Multiple values can be specified, separated by commas. Example:- TLSv1.2, TLSv1.1, TLSv1
tlsCiphers Specify the tls cipher the broker will use to negotiate during TLS Handshake. Multiple values can be specified, separated by commas. Example:- TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256
tokenSecretKey Configure the secret key to be used to validate auth tokens. The key can be specified like: tokenSecretKey=data:base64,xxxxxxxxx or tokenSecretKey=file:///my/secret.key
tokenPublicKey Configure the public key to be used to validate auth tokens. The key can be specified like: tokenPublicKey=data:base64,xxxxxxxxx or tokenPublicKey=file:///my/secret.key
tokenPublicAlg Configure the algorithm to be used to validate auth tokens. This can be any of the asymettric algorithms supported by Java JWT. RS256
tokenAuthClaim Specify the token claim that will be used as the authentication "principal" or "role". The "subject" field will be used if this is left blank