What’s New in Apache Pulsar 2.10.1

The Apache Pulsar community releases version 2.10.1! 50 contributors provided improvements and bug fixes that delivered 200+ commits. Thanks for all your contributions.

The highlight of the 2.10.1 release is introducing 30+ transaction fixes and improvements. Earlier-adoption users of Pulsar transactions have documented long-term use in their production environments and reported valuable findings in real applications. This provides the Pulsar community with the opportunity to make a difference.

This blog walks through the most noteworthy changes. For the complete list including all feature enhancements and bug fixes, check out the Pulsar 2.10.1 Release Notes.

Notable enhancements and bug fixes

Fixed ineffective load manager due to broker’s zero resource usage. PR-15314

Issue

Introduced in 2.10.0, the leader broker’s resource usage (CPU, memory, direct memory…) was always 0 when performing load balance. The root cause is that deserializing the JSON data to ResourceUsage POJO didn’t use the constructor ResourceUsage (double usage, double limit), so the percentage was always 0.

Allow users with produce/consume privileges to get topic schema. PR-15956

Issue

In earlier versions, only users with admin privileges were able to get topic schema, which made schema inconvenient to use.

Resolution

Allow users who have metadata access privileges to get topic schema. Subscribers can be from different teams, and the producers and subscribers should be able to get the topic schema instead of asking the tenant admin to do so before publishing and consuming messages.

Fixed the consumption performance regression. PR-15162

Issue

This performance regression was introduced in 2.10.0, 2.9.1, and 2.8.3. You may find a significant performance drop with message listeners while using Java Client. The root cause is each message will introduce the thread switching from the external thread pool to the internal thread poll, and then to the external thread pool.

Resolution

2.10.1 is the first version to have this issue fixed by avoiding the thread switching for each message to improve consumption throughput.

Fixed a deadlock issue of topic creation. PR-15570

Issue

This deadlock issue occurred during topic creation by trying to re-acquire the same StampedLock from the same thread when removing it. This will cause the topic to stop service for a long time, and ultimately with a failure in the deduplication or geo-replication check. The workaround is restarting the broker.

Fixed key-shared delivery of messages with interleaved delays. PR-15409

Issue

This is a regression issue introduced in 2.10.0. When delayed messages with interleaved delays occurred on a shared/key-shared subscription, many of the messages were not delivered but stayed in the backlog. The reason was that when peeking into getMessagesToReplayNow(), we could not discard the returned set due to untracked message IDs in the delayed message controller.

Optimized the memory usage of brokers.

Issue

Pulsar has some internal data structures, such as ConcurrentLongLongPairHashMap, and ConcurrentLongPairHashMap, which can reduce the memory usage rather than using the Boxing type. However, in earlier versions, the data structures were not supported for shrinking even if the data was removed, which wasted a certain amount of memory in certain situations.

Pull requests

• https://github.com/apache/pulsar/pull/15354
• https://github.com/apache/pulsar/pull/15342
• https://github.com/apache/pulsar/pull/14663
• https://github.com/apache/pulsar/pull/14515
• https://github.com/apache/pulsar/pull/14497

Resolution

Support the shrinking of the internal data structures, such as ConcurrentSortedLongPairSet, ConcurrentOpenHashMap, and so on.

What’s Next?

If you are interested in learning more about Pulsar 2.10.1, you can download and try it out now!

Pulsar Summit San Francisco 2022 will take place on August 18th, 2022. Register now and help us make it an even bigger success by spreading the word on social media!

For more information about the Apache Pulsar project and current progress, visit the Pulsar website, follow the project on Twitter @apache_pulsar, and join Pulsar Slack!

Get involved

To get started, you can download Pulsar directly or you can spin up a Pulsar cluster with a free 30-day trial of StreamNative Cloud! We also offer technical consulting and expert training to help get your organization started. As always, we are highly responsive to your feedback. Feel free to contact us if you have any questions at any time. We look forward to hearing from you and stay tuned for the next Pulsar release!