Unify Achieves Real-Time Go-To-Market Scale with Apache Pulsar and StreamNative Cloud

‍Executive Summary

• Real-Time Pipeline at Scale: Unify’s AI-driven go-to-market platform ingests tens of millions of events per day in real time, enabling instantaneous lead scoring and workflow triggers that turn growth into a science.
  
• Simplified, Reliable Architecture: By replacing batch jobs and a legacy queue system (AWS SQS) with Apache Pulsar, Unify consolidated multiple use cases (event streaming, pub-sub, and scheduling) onto one platform. This simplification improved scalability and eliminated costly cron jobs, all while providing message retention for replays and 20× faster monitoring metrics (30 seconds vs. ~10 minutes).
  
• High Leverage & Resilience: With StreamNative Cloud, Unify achieved a stable, hands-off infrastructure that easily auto-scales to demand. The asynchronous architecture ensured zero downtime even during major cloud outages, and features like delayed messages and replayable logs give the engineering team peace of mind when deploying new AI-driven features.
  

Customer Overview

Unify is a San Francisco-based, AI-native go-to-market (GTM) platform that helps revenue teams capture intent signals and engage prospects in real time. Backed by leading investors – including the OpenAI Startup Fund – Unify has rapidly grown, raising a $40 million Series B in July 2025. The company’s platform combines data integration, AI agents, and automated outreach into one system of action. From website clicks to CRM updates, Unify tracks all buyer interactions and uses AI to personalize outreach, aiming to transform growth into repeatable, scalable science.

From day one, Unify’s founding team set out to build the product on a real-time, event-driven architecture. The vision was to avoid traditional batch processing or daily jobs in favor of continuous streams of data. This approach would allow the young startup to scale more easily and deliver instant insights to sales teams. However, achieving high-throughput, real-time data flow with a lean team presented early technical challenges.

Challenges

Unify’s initial infrastructure relied on AWS SQS for message queuing, but as the platform’s ambitions grew, this approach revealed significant limitations:

• Scaling Batch Jobs: Early on, Unify realized that a pipeline built on cron-triggered batch jobs would not scale gracefully. "When you design an architecture built on jobs, the jobs break at every order of magnitude," explains Sam Waterbury, Founding ML Engineer at Unify. "It’s not simple to horizontally scale batch jobs". The team needed an event-driven model where adding consumers could handle load increases seamlessly, without reengineering pipelines at each growth step.
  
• Pub/Sub and Fan-Out: Unify’s product demanded that a single inbound event (e.g. a website visit) trigger multiple downstream actions in different services (lead scoring, AI research agent, analytics, etc.). SQS, however, is a point-to-point queue system. Implementing a publish-subscribe pattern with multiple subscribers on SQS was clunky and limited. The team sought a true pub/sub messaging system to easily fan out events to many services in parallel.
  
• No Replay or Retention: Because Unify’s AI algorithms and workflows continuously evolve, the engineers wanted the ability to reprocess past events in case of a bug fix or a new feature. SQS does not retain messages once consumed, making it impossible to "replay" historical events. This lack of message retention introduced risk — a glitch in a consumer could permanently lose valuable event data with no way to recover. Unify needed a durable log of events to enable replays and backfills for peace of mind.
  
• Operational Visibility: Monitoring and scaling on SQS proved too reactive. Metrics on queue length and throughput often lagged by up to 10 minutes in cloud dashboards, slowing down auto-scaling responses. In a dynamic environment with spiky web traffic, Unify required faster insight into backlog growth to scale consumer instances promptly and avoid delays.
  
• Lean Team & Maintenance Burden: As a small, fast-growing startup, Unify was wary of managing complex infrastructure. They evaluated Apache Pulsar (the cloud-native distributed messaging platform originally created at Yahoo), which offered the features they needed. However, self-hosting a Pulsar cluster would demand significant DevOps effort. "I spent about a day trying to self-host Pulsar," Sam admits, "and I quickly realized I didn’t want to be in the business of managing that." The team needed a managed solution that would free them to focus on product development.
  

Solution

After weighing their options, Unify migrated from SQS to StreamNative Cloud - a fully managed Apache Pulsar service – providing a real-time streaming backbone without the ops overhead. This switch addressed the startup’s challenges head-on:

• True Multi-Subscriber Streaming: With Pulsar, Unify can publish an event once and have multiple services subscribe to it in parallel. For example, when a prospect visits a website, that single event is written to a Pulsar topic and picked up simultaneously by separate consumer services: one updates the person’s intent score, another triggers an AI research agent to qualify the lead, and yet another logs the event for analytics. Pulsar’s built-in pub/sub capability elegantly handles this fan-out, something that was manual and error-prone with SQS.
  
• Built-in Durability and Replay: StreamNative Cloud retains all events even after consumers process them, allowing Unify to replay data if needed. If the team discovers a bug in an AI scoring algorithm, they can fix the code and reconsume the past events from Pulsar’s log to correct the scores. This replay feature provides a safety net that was absent before. "We wanted the ability to retain messages after processing so we could replay them if something was wrong," Sam says. "Pulsar gave us that peace of mind."
  
• Delayed Message Scheduling: Unify took advantage of an innovative Pulsar feature – delayed message delivery – to eliminate batch jobs entirely in one core use case. The platform’s intent scoring system needs to decay a lead’s score over time if no new events occur. Instead of running a scheduled job every few hours, Unify’s scoring service simply sends a message to a Pulsar topic addressed to itself with a 24-hour delay. If no new activity happens, Pulsar delivers that message back to the service the next day, prompting it to lower the score and schedule another delayed message. This cycle continues for 30 days until the score naturally falls to zero. If a new event comes in, the delayed message is canceled and the cycle restarts. By using Pulsar as a built-in scheduler, Unify eliminated the need for external cron jobs or workflow schedulers for this task. The result is an elegant, fully event-driven solution for a problem that traditionally required complex batch processing.
  
• SaaS Service & Ease of Scale: Adopting StreamNative Cloud meant Unify’s small engineering team didn’t have to worry about cluster operations, updates, or scaling the messaging layer. Pulsar’s elastic scalability and StreamNative’s 24/7 management ensure that as Unify’s data volumes grow, the messaging infrastructure seamlessly scales with it. The team can easily add consumer instances in new cloud regions or for new customers without re-architecting their pipeline. Furthermore, real-time metrics from Apache Pulsar (with roughly 30-second latency) feed into Unify’s Datadog dashboards, allowing them to auto-scale consumer pods within moments of a queue build-up — a drastic improvement over the laggy SQS monitoring.
  
• Wide Feature Set for Future Needs: Pulsar’s rich feature set (beyond pub-sub and delays, it offers geo-replication, schema registry, tiered storage, and more) gives Unify confidence that the platform can adapt to new requirements. The team is already planning to leverage key-shared subscription to route all events for a given tenant to the same consumer instance. This will improve throughput by enabling better batching and reducing database record contention. They also have topic compaction on their roadmap, which will allow them to retain a long history of events but quickly reload only the latest state per key when reprocessing (ideal for rebuilding materialized views or caches). Knowing these capabilities are readily available in Pulsar means Unify can unlock new use cases without introducing additional systems.
  

Results

By implementing StreamNative Cloud, Unify realized a robust, real-time data pipeline that underpins their AI-powered product. Key outcomes include:

• Real-Time Customer Interactions: Unify’s platform now reacts to prospect behaviors in seconds rather than hours. Website clicks, product usage events, and CRM updates flow through Pulsar and immediately trigger personalized actions (like an AI agent researching the lead or a tailored email sequence). This real-time responsiveness has enhanced lead engagement and helped generate more pipeline for Unify’s clients. Sales reps can reach out at the perfect time with relevant context, significantly improving conversion rates.
  
• Massive Scale & Performance: The Pulsar-based architecture easily handles tens of millions of events per day and scales horizontally. As Unify’s customer base grew and data volume surged, the event pipeline did not become a bottleneck. The team avoided the typical pains of batch-job systems that often fail or require re-engineering at high scale. "We’ve built a real-time system that scales without breaking a sweat, thanks to Pulsar’s consumers and queues," Sam notes. The switch to Pulsar removed the strain that SQS and batch processes were starting to show, ensuring Unify can continue to onboard large enterprise customers and higher event loads.
  
• Increased Engineering Leverage: Unify’s engineers gained a "high leverage" infrastructure where one technology addresses many needs. Instead of stitching together separate tools for queueing, pub-sub, scheduling, and reprocessing, Pulsar provided a unified platform. This simplification accelerates development of new features. The team spends less time on plumbing and more on product innovation (like refining their AI models and workflows). It aligns perfectly with Unify’s philosophy of doing more with fewer tools. As Sam puts it, "We prefer technologies that give us a lot of leverage. Pulsar’s wide feature set means one tool covers many use cases for us."
  
• Operational Resilience: The shift to an asynchronous, Pulsar-centric architecture has made Unify’s platform inherently more resilient to failures. During a recent internet outage — when services like Cloudflare and portions of Google Cloud went down — Unify’s system stayed online and continued processing events. Thanks to Pulsar’s durable storage, incoming data simply queued up during downstream outages and automatically caught up once connections were restored. End users experienced slightly delayed updates, but no data was lost and the application did not crash. This kind of graceful degradation and quick recovery is a major competitive advantage, ensuring Unify delivers a reliable experience. Moreover, with near-real-time visibility into event backlogs and consumer lag, the team can proactively manage throughput and avoid incidents before they escalate.
  
• Faster Scaling and Cost Efficiency: With Pulsar’s metrics available almost instantly, Unify configured its infrastructure to auto-scale consumer services within seconds of surging load. For example, if a viral event drives a sudden spike in website traffic, Pulsar’s topic metrics in StreamNative Cloud will reflect the growing backlog almost immediately. Kubernetes can then spin up additional consumers to drain the queue, and scale back down when the burst subsides. This responsiveness not only maintains performance but also optimizes costs by right-sizing resources in real time (no need to over-provision for worst-case traffic). In contrast, the old SQS setup might lag so much that scaling actions came minutes too late, or not at all, leading to either delays or wasted compute.
  
• Peace of Mind for Developers: Perhaps most importantly, Pulsar’s reliability and tooling have given Unify’s developers confidence in their pipeline. They know that if a bug slips through, they can rewind and reprocess data. And they trust that StreamNative Cloud will keep the system stable. "With StreamNative managing Apache Pulsar, it’s been a very stable, hands-off experience — it fits our low-risk engineering approach," says Sam Waterbury. "The ability to replay messages after they’ve been processed gives us a lot of peace of mind when we ship new code." This confidence lets the team move faster and innovate, without fearing that a small mistake will permanently drop data or take down the system.
  

Use Cases

Unify’s deployment of Apache Pulsar has unlocked a range of impactful use cases in their go-to-market platform:

• High-Throughput Data Ingestion: Pulsar serves as the central event hub for all of Unify’s data streams. Web tracking events (via JavaScript tags on clients’ sites) and CRM updates (from systems like Salesforce and HubSpot) are continuously published into Pulsar. On the busiest days, Unify processes millions of events with ease. Pulsar’s distributed log handles this firehose of data while ensuring each event is delivered to the appropriate services for processing.
  
• Real-Time Intent Scoring (with Decay): Every inbound signal – a page view, a form fill, an email click – updates Unify’s proprietary intent score for that account or lead. Those updates happen instantly via Pulsar messages. If activity ceases, Unify uses delayed events in Pulsar to gradually decrease the score over time. For example, when a lead’s score is updated, the system schedules a follow-up event in Pulsar for 24 hours later. If the lead generates no new interactions by then, the delayed message triggers and lowers the score, then schedules the next decrement. This innovative use of Pulsar’s delay feature allows scores to naturally decay over 30 days of inactivity – without any nightly batch jobs. The result is a continuously up-to-date view of customer intent, powered entirely by streaming events.
  
• AI-Driven Workflows: Unify’s "Plays" – multi-step, AI-driven sales workflows – are fueled by Pulsar events. A simple example is an automated workflow that engages a new website visitor: as soon as a visit event hits Pulsar, it triggers an AI research agent to qualify the company and fetch relevant info. Next, another service might draft a personalized outreach email. Because all these services subscribe to the event stream, the workflow unfolds in real time. Additionally, Pulsar’s speed ensures that if a prospect takes a key action (for instance, replies to an email or requests a demo), that event will instantly halt any redundant follow-ups. This prevents embarrassing overlaps (like sales contacting a lead who already responded) and lets Unify hand off hot leads to human reps at just the right moment. The end-to-end process is event-driven, delivering timely and context-aware responses that would be impossible to coordinate in a slower, batch-based system.
  
• Resilient Asynchronous Processing: Unify’s use of Pulsar has made their overall architecture more fault-tolerant. Because producers and consumers are decoupled by the Pulsar queue, a failure in one component doesn’t cascade. For example, if the CRM integration service goes down temporarily, incoming CRM events are safely buffered in Pulsar until the service is restored – with no data loss. This buffering behavior proved critical during external outages; Pulsar acted as a shock absorber that kept Unify’s internal systems from being overwhelmed or dropping data. Once the downstream systems recovered, they simply caught up on the backlog. Unify’s customers benefited from consistent uptime, and the engineering team could resolve issues without firefighting live traffic in the moment.
  
• Future Expansion with Advanced Features: As Unify continues to grow, it plans to harness more advanced Pulsar capabilities. One upcoming use is key-based ordering: partitioning topics such that all messages for a given account or tenant route to the same consumer instance. This will enable more efficient processing (e.g. aggregating events per account sequentially) and reduce database locking contention across tenants. Another planned feature is topic compaction for select event streams. In scenarios like rebuilding a lead database from event logs, compaction will allow Unify to retain a long history but quickly load only the latest state for each lead – vastly speeding up replays or backfills. These features are available out-of-the-box in Pulsar, meaning Unify can adopt them without introducing new systems. The flexibility of Pulsar ensures that Unify’s messaging infrastructure will continue to meet new demands on their path to transforming go-to-market operations.
  

By leveraging Apache Pulsar’s rich ecosystem, Unify has built a future-proof, real-time foundation for its platform. What started as a need for a better queue became a strategic advantage: an event-driven system that delivers the right information to the right service (or salesperson) at exactly the right time. Unify’s engineering choices have enabled the company to provide its customers – fast-growing sales teams – with up-to-the-second insights and automations, all on the back of a scalable, reliable streaming core.