Pulsar Summit SF 2022: Community Event Recap
We’re sad that Pulsar Summit San Francisco 2022 is over, but excited to share the event was a big hit! Let’s just say the venue was pulsing with energy. (See what we did there? 👀)
Pulsar Summit SF marked the first-ever, in-person event in North America, and it was incredible to see members of the data streaming community convene for great conversations and knowledge sharing. The turnout was impressive — the event sold out, and the team had to add tables to accommodate more people.
We want to thank the entire Apache Pulsar community and friends from other open-source communities for making the event a success. In addition to the Pulsar community, the event was well-attended by open-source partners and friends including Apache Pinot, Apache Flink, Delta Lake, Apache Beam, Apache Hudi, and ScyllaDB. Now for the highlights:
- 200+ attendees from Apple, Blizzard, IBM, Optum, Iterable, Twitter, Uber, and more!
- 20 speakers from companies like Google, AWS, and Yahoo
- 5 keynotes on Apache Pulsar and other streaming technologies like Apache Beam and Apache Pinot
- 12 breakout sessions on tech deep dives, use cases, and ecosystem talks
The talks highlighted how organizations rely on not just a single technology, but rather on an ecosystem of technologies to build out streaming use cases. The event felt truly unique because of the focus on community and representation of the broader streaming ecosystem.
Keynotes and sessions at a glance
Summit speakers covered topics on building event-driven architectures, streaming at scale, and leveraging multiple streaming technologies for a unified data stack. Here’s a quick recap of the keynotes and breakout sessions, which will be available on-demand after editing is complete. Sign up for access to the recorded content here.
Keynotes
- Microservices and event-driven architectures: Sijie Guo and Matteo Merli (StreamNative) discussed the evolution of microservices and the fundamentals of modern event-driven applications.
- Stream processing with Pulsar and Beam: Byron Ellis (Google) shared how to do stream processing at scale using Pulsar and Beam.
- User-facing analytics with Pinot: Xiang Fu (StarTree) talked about democratizing data decision-making and using Pinot for user-facing analytical applications.
- Pulsar at scale: Ignacio Alvarez (Mercado Libre) shared his company’s success with Pulsar at scale across 200M RPM and thousands of instances.
- Asynchronous messaging with Python and Pulsar: Zac Bentley (Klaviyo) discussed how the company dealt with messaging scalability challenges and built an asynchronous application framework with Python and Pulsar.
Tech Deep Dive/Use Case Breakout Sessions
- How Yahoo uses Pulsar: Ludwig Pummer and Rajan Dhabalia (Yahoo) offered insights on the challenges of cloud messaging systems and selecting Pulsar for its availability, performance, and cost.
- Broker rebalancing in Pulsar: Heesung Sohn (StreamNative) led a deep dive on broker rebalancing in Pulsar.
- Messaging redelivery in Pulsar: David Kjerrumgaard (StreamNative) walked the audience through what to do in publication redelivery scenarios in Pulsar.
- Pulsar failure scenarios: Lari Hotari (DataStax) discussed validating Pulsar’s behavior under failure conditions.
- Running Pulsar without ZooKeeper: Matteo Merli (StreamNative) shared how to leverage alternative metadata and coordination systems to move toward a ZooKeeper-less Pulsar.
- How Toast runs blue-green deploys with Pulsar and Envoy: Kai Levy and Zac Walsh (Toast) discussed Toast’s cloud-based microservice ecosystem, their journey from RabbitMQ to Pulsar, and how they use Envoy for blue-green deploys.
Ecosystem Breakout Sessions
- Building lakehouses with Delta Lake and Pulsar: Nick Karpov (Databricks) discussed the key features of Delta Lake and its expanding ecosystem.
- Lakehouses with Hudi and Pulsar: Alexey Kudinkin (Onehouse) and Addison Higham (StreamNative) gave an overview of why Apache Hudi and Pulsar work well for lakehouse architectures.
- Streaming pipelines with Flink and Pulsar: Caito Scherr (Ververica) talked about using Pulsar and Flink for a unified data stack and gave a live demo.
- Apache Kafka on Pulsar: Ricardo Ferreira (AWS) demoed Apache Kafka on Pulsar in three scenarios: microservices built for Kafka, CDC using Debezium for MySQL, and and stream processing using ksqlDB.
- Pulsar Functions with SQL: Neng Lu and Rui Fu (StreamNative) delivered a session on how SQL, Pulsar Functions, and Function Mesh work together to enable easy access to real-time data in the cloud.
- Event streaming with ScyllaDB: Peter Corless (ScyllaDB) shared his insights on distributed database design for high-performance event streaming.
More highlights: In-person networking, happy hour, and a roaming photo booth
We love the versatility of virtual events and bringing people in different locations together. But there’s nothing quite like finally meeting your new Twitter pal in person or learning from other data streaming thought leaders. And Pulsar Summit SF had plenty of opportunities to meet and learn!
According to Tim Spann, Developer Advocate at StreamNative, “The Ecosystem breakout sessions were impressive and showed the strength of Apache Pulsar when combined with lakehouses, streaming pipelines, and fast databases. Those sessions also offered validation that Pulsar belongs squarely in the middle of all modern data architectures and excels at high-performance event streaming.”
The community was very open and welcoming, and attendees happily swapped best practices and use cases. Members of the broader ecosystem were even live-tweeting and sharing their experiences throughout the day.
.@PulsarSummit was an awesome event. Amazing technology. Industry leading use cases. The collective brainpower in the room was enough to light up San Francisco. Shoutouts to @PaaSDev @riferrei \\[pictured], and the whole cast at @streamnativeio who made it possible. #PulsarSummitSF pic.twitter.com/rlj7kCAOzX— Peter Corless 🌎☮ 💛🇺🇸🇮🇪🇺🇦🌻💉 (@PeterCorless) August 20, 2022
Shoutout to technology specific conferences that really highlight platform partnerships- it’s been really cool getting some deep dives into software like @apachehudi in the context of other streaming tech (like @apache_pulsar ) #PulsarSummitSF pic.twitter.com/xQaeoAQMwe— CAITOs (@caito_200_OK) August 18, 2022
The event wrapped up with a friendly happy hour and a fun way to capture live photos — a roaming photo booth! The photo booth captured lots of photos and GIFs of people socializing and meeting fellow attendees.
During Happy Hour, we surveyed a few people and asked what three words they would use to describe the Summit. We were pleased to hear descriptors like “community” and “educational” come up repeatedly:
- Ricardo Ferreira, Developer Advocate at AWS: “I love it!”
- Jim Zucker, Solution Architect, Financial Services at Ness: “Informative, educational, and community”
- Zac Walsh, Senior Software Engineer at Toast: “Educational, community, and fun”
- Caito Sherr, Developer Advocate at Ververica: “Engaging, mellow, and organized”
More on Pulsar Summit SF
The event team recorded all keynotes and breakout sessions, and the on-demand videos will be available soon. Sign up here to get access once the videos go live.
If you’d like to hear more about Pulsar Summit San Francisco from people who attended, check out these recap resources:
Newsletter
Our strategies and tactics delivered right to your inbox