From Pulsar to Ursa: Reflecting on 3 Years of StreamNative Cloud Evolution

When we introduced StreamNative Cloud four years ago, it began as a “Managed Cloud” service—essentially what we now refer to as Bring Your Own Cloud (BYOC). At the time, we pioneered a novel approach: a control plane that deploys a portable data plane in each customer’s cloud environment. Although we didn’t call it BYOC initially, that’s precisely what it was—a new paradigm for flexible, portable cloud deployment. Today, this original concept has evolved further into what we’re calling BYOC² (Bring Your Own Cloud and Compute), reflecting our continued commitment to delivering a seamless, portable data plane.

Our journey hasn’t stopped there. Over the years, StreamNative Cloud has matured into a multi-protocol data streaming cloud service, now powered by our Ursa Engine. We’ve moved from a single-protocol offering (centered on Apache Pulsar) to a multi-protocol service that also supports Kafka and MQTT workloads as well. The innovations we built in the Ursa Engine and StreamNative Cloud are designed to address the challenges of modern cloud infrastructure – challenges we refer to as the new CAP theorem: balancing Cost, Availability, and Performance across multi-cloud and hybrid environments.

Ursa Engine: Leaderless Architecture and Lakehouse-First Storage

At the heart of our transformation is the Ursa Engine, which combines a leaderless architecture with a lakehouse-centric storage model to optimize performance, scalability, and cost.

Leaderless Architecture

A leaderless architecture removes single-topic leadership and leverages object storage as shared storage for high-throughput, latency-relaxed use cases. As data streaming architectures evolve from leader-based to leaderless models, producers and consumers no longer need to traverse Availability Zones to write or read data from remote brokers. Simultaneously, shared storage eliminates much of the replication overhead.

This shift simplifies cluster management and operations and reduces overall networking costs, thereby lowering total infrastructure expenses. It also enables cost-effective real-time data processing without the networking overhead often seen in leader-based architectures such as vanilla Pulsar, Kafka, and RedPanda.

Lakehouse-First Storage

Ursa’s storage layer directly employs lakehouse formats (e.g., Delta Lake, Iceberg) to persist data in object storage, eliminating additional ETL steps when moving from streaming data to table data. By aligning real-time streaming with lakehouse storage, organizations can significantly reduce data movement overhead and often achieve substantial cost savings. This architecture also simplifies downstream analytics because data remains in lakehouse-compatible formats from the outset.

One System for Stream-Table Duality

In most data ecosystems, engineering teams juggle multiple systems for ingestion, processing, and storage. Ursa’s design aims to reduce this complexity by using lakehouse storage as the central engine. This tight integration means:

• Fewer ETL pipelines to maintain
• Less data duplication
• Lower latency between ingestion and query

Ultimately, this connected, end-to-end approach better serves AI and machine learning workloads that require continuous, real-time data flows.

Flexible Deployment Options: Serverless, BYOC, and Beyond

Different use cases demand different deployment strategies. StreamNative Cloud now provides four main options, each refined by our portable data plane:

• Serverless
  Spin up and scale real-time applications without manual infrastructure management.
• Dedicated
  Reserve dedicated resources in the public cloud for predictable, high-performance workloads.
• BYOC (Bring Your Own Cloud)
  Deploy our portable data plane directly in your own cloud account, with StreamNative providing expert management of your Pulsar and Kafka-compatible clusters.
• Private Cloud
  Install StreamNative Cloud on-premises or in a private environment to maintain complete control and meet stringent security mandates.

This evolution—from a standalone “Managed Pulsar” service to a fully portable, multi-protocol streaming solution—reflects our commitment to offering engineering teams maximum flexibility, while retaining visibility through robust tooling, observability, and enterprise-grade support.

StreamNative Cloud on All Major Clouds

Beyond these flexible deployment models, StreamNative Cloud is also available across the three leading public cloud providers—AWS, GCP, and Azure. Even better, you can procure StreamNative Cloud directly through their respective marketplaces, simplifying your purchasing process and making it easy to start with just one click.

Whether you need data streaming for short-term projects or enterprise-grade, long-term deployments, you can quickly spin up StreamNative Cloud wherever your data resides. This seamless availability helps businesses accelerate time to value by combining real-time data ingestion with a streamlined procurement process.

Security & Compliance

Security and compliance are foundational to everything we build at StreamNative. Our platform has evolved to provide a suite of enterprise-grade features that ensure your data remains protected and compliant across a range of regulatory requirements and industry best practices:

• Single Sign-On (SSO) & Role-Based Access Control (RBAC)
  StreamNative Cloud integrates with your identity provider for seamless SSO and offers granular RBAC to control user permissions. This guarantees that only authorized personnel can access your data and services.
• Encryption at Rest and in Transit
  We support encryption at rest with bring your own key (BYOK) integrations and secure transmission via TLS. This ensures that data remains encrypted on disk and during network communication, safeguarding against unauthorized access.
• End-to-End Encryption
  For additional privacy, you can enable end-to-end encryption, meaning only your applications hold the keys needed to decrypt data. This ensures that even StreamNative Cloud cannot access the contents of your messages.
• Data Sovereignty with BYOC
  By allowing you to deploy the data plane in your own cloud environment, StreamNative Cloud ensures your data never leaves your chosen infrastructure. This is crucial for meeting data sovereignty requirements and maintaining complete control over where your data resides.
• Global Compliance Standards
  StreamNative Cloud meets a wide array of compliance requirements, including SOC 2, ISO 20007, PCI, HIPAA, and GDPR. With these certifications, you can trust that our platform aligns with the highest standards in security and data protection.

Whether you’re operating in a highly regulated industry or simply value robust security controls, StreamNative Cloud delivers the confidence and compliance you need to accelerate innovation—without compromising on data protection.

Strategic Partnerships

StreamNative partners with industry leaders such as Ververica, Snowflake, and Google, among others, to enhance its data streaming and processing capabilities while providing robust support for open lakehouse formats like Delta Lake and Apache Iceberg. Additionally, StreamNative partners with esteemed System Integrators, including CalSoft, Nexaminds, and Nuaav, to deliver comprehensive support for customers across their entire data platform journey. 

Success Stories: Powering Mission-Critical Apps

Our customers span a variety of industries, but they all share one thing in common: the need for a platform that handles mission-critical and transactional workloads in real time. A few highlights from our success stories and case studies

• Cisco: Manages lifecycle operations for IoT devices on a massive scale (35,000 enterprises, 245 million+ connected devices), relying on StreamNative Cloud for high-volume, ultra-reliable telemetry.
• Iterable: Processes billions of daily events to facilitate hyper-personalized real-time marketing and customer engagement.
• FICO: Builds scalable, secure data pipelines for fraud detection and real-time observability, supported by StreamNative’s robust architecture.
• Innerspace: Uses Pulsar to analyze sensor data, improving workplace safety and enabling data-driven optimization.
• TEG: Modernized its data infrastructure to better match shippers and carriers, reducing latency and vastly improving the end-customer experience.

StreamNative Cloud: Your Data Streaming Partner in the age of AI

We’ve delivered significant innovations over the past few years, but where are StreamNative and data streaming headed in the AI era? The key lies in building end-to-end AI pipelines that integrate ingestion, analytics, and machine learning workflows under one platform. Through our Ursa Engine, StreamNative Cloud will continue to drive the convergence of data streaming and lakehouse systems. This includes native compatibility with leading lakehouse formats (like Iceberg and Delta Lake) and automatic, Kafka-compatible ingestion pipelines, significantly reducing redundant ETL tasks and data movement. As a result, engineering teams can focus on insights instead of infrastructure.

Additionally, StreamNative Cloud is designed to serve both real-time and AI-driven workloads seamlessly, ensuring low-latency data delivery for AI applications that demand quick, accurate decisions. Ultimately, StreamNative Cloud is more than just a Pulsar or Kafka platform—it’s the critical link connecting real-time data ingestion with advanced analytics and machine learning. By uniting Apache Pulsar’s robust messaging model with Ursa’s Kafka compatibility, lakehouse-first architecture, and flexible deployment through portable data planes, we empower businesses to simplify data streaming, accelerate real-time analytics, and power next-generation AI solutions.

Let’s build the future—together.

‍