Announcing: The Pulsar PMC Published The 2020 Apache Pulsar User Survey Report!

Overview
Get started
Install and upgrade
Configure
Overview
Pulsar core
Control center
Pulsar Manager
Prometheus
Grafana
Alertmanager
Tool
Secure
Manage and monitor
Connect
Process
Release notes

Configure Prometheus

Prometheus is a component of "control center" module in StreamNative Platform.

To display the metrics for StreamNative Platform with Prometheus, configure your Prometheus server as below.

By default the config file is located in ${PLATFORM_HOME}/etc/sn-prometheus/prometheus.yml.

Note: The sn-prometheus provides standalone.yml.bak and cluster.yml.bak in ${PLATFORM_HOME}/etc/sn-prometheus/. If you want to customize more metrics information, you can change it in ${PLATFORM_HOME}/etc/sn-prometheus/.

  1. (Only for cluster) Attach your Prometheus service to an extra-label cluster.

    The cluster name is aligned with the PULSAR_CLUSTER name you have provided to the Grafana dashboard.

    global:
      ...
      external_labels:
        cluster: <your-cluster-name>
  2. (For standalone and cluster) Make sure the job name of each component is the same with the ones in this dashboard.

  • job proxy: the machines that run pulsar proxies.
  • job broker: the machines that run pulsar brokers.
  • job bookie: the machines that run bookies.
  • job zookeeper: the machines that run zookeeper.
  • job node_metrics: all the machines of the pulsar cluster.
  1. (For standalone and cluster) Generate a Prometheus config file using one of the following methods.
  • Method 1

    Copy the Prometheus configuration template to your current directory and rename it to prometheus.yml, then replace {{ STANDALONE_HOST }} with your IP address of the machine running Pulsar standalone.

    Tip

    • For Prometheus configuration template (standalone), see here.
    • For Prometheus configuration template (cluster), see here.
  • Method 2

    Install j2cli. j2cli is a command-line tool for templating Jinja2 template files. You can use j2cli to generate a Prometheus config file from the standalone template.

    • MacOS

      $ STANDALONE_HOST="$(ifconfig | grep "inet " | grep -v 127.0.0.1 | awk '{ print $2 }')" j2 prometheus/standalone.yml.template > ${PLATFORM_HOME}/etc/sn-prometheus/prometheus.yml
    • Ubuntu

      Set STANDALONE_HOST as below.

      $ STANDALONE_HOST="$(ifconfig | grep "inet " | grep -v 127.0.0.1 | awk '{ print $2 }' | awk -F ':' '{ print $2 }' | awk 'NR==2')" j2 ${PLATFORM_HOME}/etc/sn-prometheus/prometheus.yml

    Note

    If fail to start Grafana, you can set the IP manually.

  1. (For standalone and cluster) Run Prometheus with the generated configuration file.

    ${PLATFORM_HOME}/bin/sn-prometheus

    After running Prometheus successfully:

    • For standalone: you can access to the Prometheus website on http://localhost:9090/targets where you can see Prometheus detecting all pulsar components as below.

    • For cluster, you can access to the Prometheus website on http://prometheus-server-ip:9090 where you can see Prometheus detecting all pulsar components as below.