What’s New in Apache Pulsar 2.6.4
We are excited to see the Apache Pulsar community has successfully released the 2.6.4 version! 10 contributors provided improvements and bug fixes that contributed to 16 PRs.
- Broker no longer delivers old messages after a topic is closed (#8634)
- AWS credentials are refreshed after expiry (#9387)
- Pulsar identifies when individual message deletes cause an unsynced cursor (#9732)
This blog walks through the most noteworthy changes. For the complete list including all enhancements and bug fixes, check out the Pulsar 2.6.4 Release Notes.
C++ client supports multiple topic subscriptions across multiple namespaces (#9520)
Previously, you could not subscribe to different topics on different namespaces.
- Move the check for namespace in MultiTopicsConsumerImpl to PatternMultiTopicsConsumerImpl that uses a regex subscription.
- Fix the existing tests for subscriptions on topics across different namespaces.
Notable bug fix
Pulsar guarantees security for clients using JWT (#9172)
Previously, it was possible for attackers to connect to Pulsar instances because the signature of the JWT was not validated when the token was set to none.
Modified JWT to use parseClaimsJws instead of parse to get the token objects. Now, parseClaimsJws guarantees the correct security model for parsing signed JWTs.
Pulsar identifies when individual message deletes cause an unsynced cursor (#9732)
Previously, cursors were not being flushed when acknowledgements caused a dirty cursor. Instead of deleting the acknowledged messages, messages were redelivered.
Fixed code to mark the individual acknowledgements and automatically trigger the flush of dirty cursors.
Pulsar can expire a range of messages (#9083)
Previously, only a single message expired after an expiry check. As a result, many expired messages remained in a subscription and were delivered to consumers after the expiry time.
Modified OpFindNewest to jump to a valid position, which allows PersistentMessageExpiryMonitor to find the best range of messages to expire.
Pulsar allows manual (forced) topic deletion after removing non-durable subscriptions (#7356)
Previously, during the removal of non-durable subscriptions, there was a race condition that left a topic in a state where you could not delete it until it was unloaded or reloaded.
Fixed the race condition by setting the topic fence before performing any delete operations and reverting the topic state after the delete operations.
Broker no longer delivers old messages after a topic is closed (#8634)
Previously, it was possible to re-deliver very old messages if a topic was not gracefully closed. The cursor rolled back to the last persisted position and triggered the re-delivery of those messages.
Fixed the redelivery of messages by setting a time-bound period after which all cursor updates are flushed on the disk.
Batch index acknowledgement data is no longer persisted (#9504)
Previously, the batch index acknowledgement data persisted because batchDeletedIndexInfoBuilder generated the batch index acknowledgement data but did not clear the current set before adding the delete set.
Fixed by clearing the delete set before adding a new delete set.
Closed ledger deletes after expiration (#9136)
Previously, a closed ledger (with no incoming traffic) could fail to delete after expiring because the read position of the cursor still points to the last entry of the closed ledger.
Updated behavior when closing the current ledger. Now, when the cursor's mark-delete position points to the last entry of the current ledger, the read position is moved to the newly created ledger.
AWS credentials are refreshed after expiry (#9387)
Previously, expired AWS credentials were reused. With the refactor of Azure support, a regression occurred where the AWS credentials are fetched once and then used through the entire process.
The AWS credential provider chain takes care of the credential refresh. When integrating with JClouds, you still need to return a new set of credentials each time.
Compression applied during schema preparation (#9396)
Previously, compression was not applied during deferred schema preparation and the consumer could receive an uncompressed message and then fail.
Fixed by enforcing compression during the schema preparation.
To get started, you can download Pulsar directly or you can spin up for a Pulsar cluster on StreamNative Cloud with a free 30-day trial of StreamNative Cloud in which Pulsar 2.6.4 changes are shipped! Moreover, we offer technical consulting and expert training to help get your organization started. As always, we are highly responsive to your feedback. Feel free to contact us if you have any questions at any time. Look forward to hearing from you and stay tuned for the next Pulsar release!