Highlights from The Pulsar Virtual Summit North America 2021

Hosted by StreamNative and Splunk, the Pulsar Virtual Summit North America 2021, took place on June 16-17th. This 2-day event was packed with Pulsar project updates, ecosystem news, and insights into some of the largest and most exciting Pulsar deployments around the globe.

Since it became a top-level Apache Software Foundation project in 2018, the community has seen an influx of adoption as companies look to Apache Pulsar's cloud-native capabilities, unified messaging and streaming, and super-set of built-in features.

Attendees had the chance to learn from Pulsar committers, contributors, experts, and adopters. Whether you were a Pulsar veteran or a newbie, the event had something for you. Below we provide some of our favorite highlights from the summit.

Summit by the Numbers

Pulsar Community by the Numbers

Major Project Updates

Last week was an exciting week for the Apache Pulsar community. Not only did the Summit happen, but the Pulsar PMC also announced the release of Pulsar 2.8.0. Summit keynotes and talks from the Technology Deep Dive Stage provided a closer look at some of the most significant project updates. We highlight these below:

• Apache Pulsar: Why Unified Messaging and Streaming Is the Future: In this keynote, Matteo Merli and Sijie Guo dive into the landscape of unified messaging and streaming, how Pulsar helps companies achieve this vision, and what the future of Pulsar will look like, including the newly released v 2.8.0.
• Exactly-Once Made Easy: Transactional Messaging In Apache Pulsar: In this session, Sijie Guo and Addison Higham discuss Pulsar transaction and how it can be applied to Pulsar Functions and other processing engines to achieve transactional event streaming.
• Replicated Subscriptions: Taking Geo-Replication to the Next Level: In this session, Matteo Merli, explores various patterns of cluster failover, when it is appropriate to use them and the tradeoffs found in each approach.
• Take Kafka-on-Pulsar to Production at Internet Scale: Improvements Made for Pulsar 2.8.0: In this talk, Yunze Xu and Aloys Zhang introduce how KoP has been improved for Pulsar 2.8.0, including the new implementation of KoP offset, the Kafka entry formatter that improves the performance, and additional changes that improve KoP’s stability and Kafka protocol compatibility.
• Advanced Stream Processing with Flink and Pulsar: In this talk, Till Rohrmann and Addison Higham discuss how Flink allows for ambitious stream processing workflows and how using Pulsar and Flink together enables new capabilities that push forward the state-of-the-art in streaming.
• Function Mesh: Complex Streaming Jobs Made Simple: In this talk, Neng Lu and Rui Fu provide a walkthrough of the Function Mesh, including its design, implementation, use cases, and examples. This session helps users understand how Function Mesh can be used to simplify complex streaming solutions.

Key Use Cases

The Summit was jam-packed with groundbreaking Pulsar use cases, from Karthik Ramasamy’s session on Scaling Apache Pulsar to 10 Petabytes/Day to Intuit’s talk on Building the Next-Generation Messaging Platform on Pulsar. More on these and other favorites below:

• Splunk’s Karthik Ramasamy presented Scaling Apache Pulsar to 10 Petabytes/Day. In this talk, we learn how Splunk helped a flagship customer scale a Pulsar deployment to handle 10 PB/day in a single cluster.
• Intuit’s Madhavan Narayanan, Sajith Sebastian, Amit Kaushal, Gokul Sarangapani present: Building the Next-Generation Messaging Platform on Pulsar at Intuit. In this talk we learn how they use Pulsar for their highly distributed, multi-cluster, multi-region messaging platform to serve the queuing use-cases of its applications and services.
• Narvar’s Ankush Goyal shares How Narvar Uses Pulsar to Power the Post-Purchase Experience. In this session we learn about Narvar’s platform, which is built with pub-sub messaging at its core, making reliability, scalability, maintainability, and flexibility business critical features. Ankush shares why Narvar adopted Pulsar and how Narvar is leveraging Pulsar today.
• MicroFocus’s Srikanth Natarajan presents Why Micro Focus Chose Pulsar for Data Ingestion. In this talk, we learn about Micro Focus’ adoption of Pulsar, including the lessons learned, and the help that Micro Focus received from a development support partner in their Pulsar journey.
• Iterable’s Tom Wang and Thomas Kim present Migrating a Billion Transactions a Day to Apache Pulsar. In this talk we learn how Iterable, a marketing automation SaaS that operates at a vast scale, processes a billion queue transactions per day.
• Verizon Media’s Rajan Dhabalia and Ludwig Pummer present on Security and Multi-tenancy with Apache Pulsar in Yahoo!. In this talk we learn how Verizon Media uses different multi-tenancy dimensions of Apache Pulsar to serve multiple use cases and applications on a shared pulsar cluster.

Summit News

In addition to the great content, there were other project updates announced during the summit:

• Pulsar Hackathon Winners: The winners of Pulsar Hackathon 2021 were announced at the summit. You can learn more about the top projects and view the video submissions here.
• CFP for Pulsar Summit Europe 2021: The CFP for Pulsar Summit Europe 2021 is now open. Submit a talk!

We want to thank our event hosts, community sponsors, and speakers for making this an unforgettable event. We also want to say thank you to everyone who attended! We hope you enjoyed it as much as we did. And, don’t worry if you missed the live event. The recordings and slides are available.

Stay Connected

If you’re not signed up for the StreamNative newsletter, sign up today.

‍