Apache Pulsar is a distributed messaging platform and a fast-growing alternative to Kafka. Apache SkyWalking is a popular application performance monitoring tool for distributed systems, specially designed for microservices, cloud-native, and container-based (Docker, K8s, and Mesos) architectures.
Message tracing is a very useful feature which helps engineers troubleshoot problems related to message publishing and receiving.
This tutorial shares how to track Pulsar messages by using SkyWalking.
Before getting started, make sure you have installed Git, JDK 8, Maven 3, and Pulsar (cluster or standalone). If you do not have an available Pulsar, follow the instruction (http://pulsar.apache.org/docs/en/standalone/) to install.
Build Pulsar agent from SkyWalking source
The Apache Pulsar agent will be officially released in the SkyWalking 6.5.0, since it has not been released yet, you need to build a Pulsar agent from the source of SkyWalking.
1. Download the SkyWalking source and build the Pulsar agent plugin.
By executing the HTTP request above, some traces are created in SkyWalking. Let’s go to the SkyWalking UI to check them.
It shows there are 2 endpoints, 1 service, and 1 MQ.
It shows that a user sends a request to the web service (that is, your test web application) and the web service sends to and receives messages from a Pulsar broker.
It shows the trace details of each request.
Currently, the Pulsar agent plugin has 3 types of spans as below:
Producer span, which records messages sent by producers.
Producer callback span, which records messages are already sent.
Consumer span, which records messages are received by consumers.
As you can see, SkyWalking UI is pretty cool! If you are still worried about how to track Pulsar messages, try this integration of Pulsar and SkyWalking.
Thanks for the SkyWalking community who gives me a lot of help for the integration of Pulsar and SkyWalking.
Penghui Li is passionate about helping organizations to architect and implement messaging services. Prior to StreamNative, Penghui was a Software Engineer at Zhaopin.com, where he was the leading Pulsar advocate and helped the company adopt and implement the technology. He is an Apache Pulsar Committer and PMC member. Penghui lives in Beijing, China.