Announcing: The Pulsar PMC Published The 2020 Apache Pulsar User Survey Report!

Get started
Install and upgrade
Manage and monitor
Pulsar connector
Get started
Release notes

Get started

In this guide, you will learn how to run KoP plugin in StreamNative Platform to serve requests from a Kafka client.


Check the following requirements before using KoP.

This example assumes that you will use local machine to start StreamNative Platform.

Quick start

Configure and start StreamNative Platform

KoP is a plugin with protocol name "kafka" in Pulsar and it is loaded when Pulsar broker starts.

  1. Decompress the downloaded .tar.gz or .zip file, and you can see the following directories.

    ├── README
    ├── bin // Store executable files provided by StreamNative Platform.
    ├── etc // Store related configuration file information.
    ├── lib // Provide systemd services. When the service is down, it restarts automatically. 
    └── share // Store dependent files.
  2. Set the environment variable for the StreamNative Platform directory.

    $ export PLATFORM_HOME=<path-to-streamnative>
  3. Check the configuration of KoP.

    As mentioned previously, KoP module is loaded along with the Pulsar broker. You need to configure the Pulsar broker to run the KoP protocol handler as a plugin, that is, add the following properties in the Pulsar configuration file, in ${PLATFORM_HOME}/etc/pulsar/standalone.conf.

    Property Set it to the following value Default value
    messagingProtocols kafka null
    protocolHandlerDirectory Location of KoP NAR file ${PLATFORM_HOME}/share/java/pulsar/protocols



    By default, these values are already set in StreamNative Platform, and kop.nar has already been placed under ${PLATFORM_HOME}/share/java/pulsar/protocols.

  4. Add the StreamNative Platform bin directory to your path.

    $ export PATH=${PLATFORM_HOME}/bin:$PATH
  5. Start StreamNative Platform using Supervisord.


    In the log of broker, after the StreamNative Platform is started successfully, you should find a log similar to this:

    12:22:22.945 [main] INFO - Successfully loaded protocol handler for protocol `kafka`

Verify with Kafka client.

  1. Download the Kafka 2.0.0 release and untar it.

    tar -xzf kafka_2.11-2.0.0.tgz
    cd kafka_2.11-2.0.0
  2. Use a Kafka producer and a Kafka consumer to verify.

    In Kafka’s binary, there is a command-line producer and consumer. These steps verify Kafka producer send message to Pulsar broker, and Kafka consumer get message from Pulsar broker.

    Run the command-line producer and send a few messages to the server.

    > bin/ --broker-list --topic test
    This is a message
    This is another message

    Kafka has a command-line consumer dumping out messages to standard output.

    > bin/ --bootstrap-server --topic test --from-beginning
    This is a message
    This is another message

More information