Jun 12, 2022
5 min read

What’s New in Apache Pulsar 2.10.1

Penghui Li
Engineering Lead of Messaging Team, StreamNative
Jun Ma

The Apache Pulsar community releases version 2.10.1! 50 contributors provided improvements and bug fixes that delivered 200+ commits. Thanks for all your contributions.

The highlight of the 2.10.1 release is introducing 30+ transaction fixes and improvements. Earlier-adoption users of Pulsar transactions have documented long-term use in their production environments and reported valuable findings in real applications. This provides the Pulsar community with the opportunity to make a difference.

This blog walks through the most noteworthy changes. For the complete list including all feature enhancements and bug fixes, check out the Pulsar 2.10.1 Release Notes.

Notable enhancements and bug fixes

Fixed ineffective load manager due to broker’s zero resource usage. PR-15314

Issue

Introduced in 2.10.0, the leader broker’s resource usage (CPU, memory, direct memory…) was always 0 when performing load balance. The root cause is that deserializing the JSON data to ResourceUsage POJO didn’t use the constructor ResourceUsage (double usage, double limit), so the percentage was always 0.

Allow users with produce/consume privileges to get topic schema. PR-15956

Issue

In earlier versions, only users with admin privileges were able to get topic schema, which made schema inconvenient to use.

Resolution

Allow users who have metadata access privileges to get topic schema. Subscribers can be from different teams, and the producers and subscribers should be able to get the topic schema instead of asking the tenant admin to do so before publishing and consuming messages.

Fixed the consumption performance regression. PR-15162

Issue

This performance regression was introduced in 2.10.0, 2.9.1, and 2.8.3. You may find a significant performance drop with message listeners while using Java Client. The root cause is each message will introduce the thread switching from the external thread pool to the internal thread poll, and then to the external thread pool.

Resolution

2.10.1 is the first version to have this issue fixed by avoiding the thread switching for each message to improve consumption throughput.

Fixed a deadlock issue of topic creation. PR-15570

Issue

This deadlock issue occurred during topic creation by trying to re-acquire the same StampedLock from the same thread when removing it. This will cause the topic to stop service for a long time, and ultimately with a failure in the deduplication or geo-replication check. The workaround is restarting the broker.

Fixed key-shared delivery of messages with interleaved delays. PR-15409

Issue

This is a regression issue introduced in 2.10.0. When delayed messages with interleaved delays occurred on a shared/key-shared subscription, many of the messages were not delivered but stayed in the backlog. The reason was that when peeking into getMessagesToReplayNow(), we could not discard the returned set due to untracked message IDs in the delayed message controller.

Optimized the memory usage of brokers.

Issue

Pulsar has some internal data structures, such as ConcurrentLongLongPairHashMap, and ConcurrentLongPairHashMap, which can reduce the memory usage rather than using the Boxing type. However, in earlier versions, the data structures were not supported for shrinking even if the data was removed, which wasted a certain amount of memory in certain situations.

Pull requests

Resolution

Support the shrinking of the internal data structures, such as ConcurrentSortedLongPairSet, ConcurrentOpenHashMap, and so on.

What’s Next?

If you are interested in learning more about Pulsar 2.10.1, you can download and try it out now!

Pulsar Summit San Francisco 2022 will take place on August 18th, 2022. Register now and help us make it an even bigger success by spreading the word on social media!

For more information about the Apache Pulsar project and current progress, visit the Pulsar website, follow the project on Twitter @apache_pulsar, and join Pulsar Slack!

Get involved

To get started, you can download Pulsar directly or you can spin up a Pulsar cluster with a free 30-day trial of StreamNative Cloud! We also offer technical consulting and expert training to help get your organization started. As always, we are highly responsive to your feedback. Feel free to contact us if you have any questions at any time. We look forward to hearing from you and stay tuned for the next Pulsar release!

Penghui Li
Penghui Li is passionate about helping organizations to architect and implement messaging services. Prior to StreamNative, Penghui was a Software Engineer at Zhaopin.com, where he was the leading Pulsar advocate and helped the company adopt and implement the technology. He is an Apache Pulsar Committer and PMC member. Penghui lives in Beijing, China.
Jun Ma
Jun Ma is a technical writer from StreamNative, focusing on the Pulsar open-source documentation. Before joining StreamNative, she has been working in the technology and engineering fields across multi-national companies. She is keen to help users understand complex concepts and bring valuable content to their information experience.

Newsletter

Our strategies and tactics delivered right to your inbox

Thank you! Your submission has been received!
Oops! Something went wrong while submitting the form.
Pulsar Releases
Apache Pulsar Announcements