General Availability for Pulsar Functions on All New Clusters in SN Cloud

Pulsar Functions

At StreamNative, we constantly strive to innovate and give you better tools to suit your streaming and messaging needs. Pulsar Functions have always stood out as one of those tools, providing an efficient and versatile solution for lightweight stream processing. Designed to operate seamlessly atop Pulsar, Pulsar Functions provide a lightweight way to consume messages from one or more topics, apply sophisticated user-defined logic, and then publish the processed messages to other topics.

For those who might be unfamiliar, Pulsar Functions offer an array of benefits:

• Seamlessly Integrated with Pulsar: Being natively integrated with Apache Pulsar, Pulsar Functions allow you to tap into the power of Pulsar with ease.
• Lightweight: With Pulsar Functions, developers can focus on their business logic without setting up an elaborate stream processing framework.
• Flexibility with Language Choices: Pulsar Functions can be written in Java or Python, with more options on the way, so you can use the language you are most comfortable with.

Pulsar Functions On StreamNative Cloud

While other Stream Processing Engines such as Apache Flink, Apache Spark, Apache Storm, and Apache Heron have undeniably carved a niche in the stream processing world, they often come with the overhead of prolonged ramp-up time, complex configurations, deployment challenges, and extra cost. On the other hand, Pulsar Functions offers a more integrated solution designed from the ground up to work in harmony with Pulsar.

That’s why we are excited to announce the General Availability of Pulsar Functions on StreamNative Cloud for all new Hosted and BYOC clusters.

What does this mean for you?

• Serverless Computing: With Pulsar Functions seamlessly integrated into StreamNative Cloud, there is no need to concern yourself with setting up and managing a separate stream processing cluster. Everything your Pulsar Functions need is already set up by StreamNative Cloud.
• Simplified Management: The process of managing and monitoring your Pulsar Functions has become significantly simplified. You can effortlessly submit or manage Functions using Terraform or pulsarctl, and conveniently access Function details and logs directly from the StreamNative Console.
• Continued Support: At StreamNative, we remain committed to providing top-notch support for all your Pulsar Functions needs. Whether you represent an enterprise or a digital-native startup, our dedicated team is readily available to offer guidance and assistance with setting up and running Pulsar Functions.
• pfSQL (Public Preview): pfSQL allows users to write Pulsar Functions using a SQL-like language, providing a rapid and lightweight stream processing method. In its first iteration, pfSQL will support filtering, routing, transformation queries, built-in user-defined functions (UDFs), and query result preview functionality, allowing users to inspect stream data for data analysis use cases. Check out our docs to learn more!

What’s next for Pulsar Functions on StreamNative

While we’re excited to announce GA for Pulsar Functions, we’re not stopping there. We’re continuing to innovate on Functions and have a lot of upcoming updates coming to Functions on StreamNative Cloud.

• Stateful Functions (coming soon): functions that can hold state, rather than requiring all rules to be preprogrammed, are extremely useful for advanced and even intermediate stream processing use cases. We are building this functionality to support even more sophisticated uses of Pulsar Functions.
• Autoscaling (coming soon): scaling Functions horizontally and vertically is important for your application’s ability to respond to greater traffic, and for cost savings when there is less traffic. We are working on a robust, configurable solution allowing you to scale your Pulsar Functions up and down based on your changing needs.
• Generic Runtime (coming soon): we want to support writing Pulsar Functions in even more languages than we do now, so we’re working on a generic runtime for functions, which will support Node.js, web assembly, and many more in the future.

Conclusion

With Pulsar Functions generally available, it’s now vastly easier to do lightweight stream processing on a StreamNative Pulsar cluster. There’s no separate computing cluster you need to set up, you can easily deploy Pulsar Functions using Terraform or pulsarctl, and you can see logs and exceptions directly within the StreamNative Console. On top of that, we’re working on a wide variety of upcoming features that will let you do even more: writing stateful functions, scaling them up and down in response to traffic, and writing them in a wider variety of languages.

To learn more about Pulsar Functions, check out our YouTube playlist, which walks you through writing and deploying a Pulsar Function and our docs on them.

If you’d like to try out Pulsar Functions, contact our sales team.

Happy streaming!