Optimize and Scale: How Kafka-on-StreamNative Transforms Your Data Streaming Platform

We are very excited to announce the general availability of Kafka-on-StreamNative (KSN). Kafka-on-StreamNative (KSN) allows you to migrate your Kafka workloads to the ONE StreamNative Platform without rewriting client code. It promises to build upon your investments in Kafka with Pulsar's enterprise capabilities. Using KSN, you can quickly and cost-effectively modernize your Kafka-based data infrastructure, achieving a truly scalable, highly available, multi-tenant data streaming platform.

Understanding KSN and the ONE StreamNative Platform

Kafka-on-StreamNative (KSN) brings native Apache Kafka protocol support to Apache Pulsar by introducing a Kafka protocol handler on Pulsar brokers. You can migrate your existing Kafka applications and services to Pulsar without modifying client code. This enables Kafka applications to leverage Pulsar’s powerful features, such as:

• Streamlined operations with enterprise-grade multi-tenancy
• Simplified operations with a rebalance-free architecture
• Automated topic rebalancing
• Infinite event stream retention with Apache BookKeeper and tiered storage
• Serverless event processing with Pulsar Functions

Kafka-on-StreamNative supports all enterprise features, including KStreams, KSQL, KTables with Topic Compaction, Schema Registry for the Java Client, and Kerberos Authentication for Kafka Clients.

The ONE StreamNative platform is a modern data streaming platform that uses Apache Pulsar and offers built-in multi-tenancy, geo-replication, and tiered storage.

The ONE StreamNative Platform is available in three deployment models:

1. Hosted - A fully managed SaaS offering
2. Bring-Your-Own-Cloud (BYOC) - Fully managed in your public cloud
3. Private Cloud - On-premises in your private cloud

‍

Kafka challenges addressed by KSN

1. Increased downtime with Kafka: Kafka users need to repartition their cluster when they reach the storage limit, which results in downtime. With KSN, you run your Kafka workloads more efficiently on Apache Pulsar without any migration headaches. Built-in multi-tenancy and broker auto-rebalancing allow you to scale Kafka workloads without worrying about maintenance overhead.
2. Manual HA/DR: Kafka users need to manually configure geo-replication to replicate data in multiple regions for HA/DR and higher uptime. With KSN, real-time data is now part of all your mission-critical applications. You can leverage built-in geo-replication functionality to replicate your Kafka workloads seamlessly around the globe.
3. High bills: Kafka users often get very high bills due to unpredictable workloads that result in over provisioning, and do not get the right ROI from Kafka investment. With KSN, you dramatically reduce data streaming costs and management complexities by consolidating 10’s of Kafka clusters into a single Pulsar cluster.

What else makes KSN stand out?

1. Scalability: One of the key advantages of Pulsar is its ability to scale effortlessly. With KSN, you can tap into Pulsar's horizontally scalable architecture, enabling you to handle massive volumes of data without compromising on performance.
2. Multi-Tenancy: Pulsar's built-in support for multi-tenancy allows different teams or applications to share the same cluster securely. KSN extends this capability to Kafka users, providing isolation and resource management for diverse workloads.
3. Tiered Storage: Pulsar's tiered storage architecture, which seamlessly integrates with cloud object stores like Amazon S3 or Azure Blob Storage, enables cost-effective long-term data retention. KSN inherits this feature, offering organizations flexibility in managing data retention policies and reducing storage costs.
4. Compatibility with Kafka Ecosystem: KSN maintains compatibility with Kafka's APIs, including producer and consumer APIs, Kafka Connect, and Kafka Streams. This allows existing Kafka applications and tools to seamlessly integrate with Pulsar, minimizing migration efforts and ensuring a smooth transition.

‍

Getting started with KSN

Experience KSN with a free trial ($200 credit) by visiting our pricing page. Additionally, enhance your skills with our short training course, 'Kafka on StreamNative (KSN) for Kafka Developers'. Explore the comprehensive KSN documentation to delve deeper into our platform.

Finally, I will be hosting a webinar on April 2nd 9amPT with my colleague Dustin to demonstrate this product.

For further inquiries, don't hesitate to contact us.

‍