TL;DR
Real-time customer-facing analytics is crucial for maintaining a competitive edge, yet traditional systems struggle with slow queries and complex pipelines. Apache Pulsar and StarRocks offer a solution by enabling sub-second queries, real-time updates, and simplified stream processing. This approach reduces infrastructure costs and improves user experience, as demonstrated by industry leaders like Naver and Airbnb.
Opening
Imagine launching a new feature only to find out it takes two to three days to update due to data processing bottlenecks. This is the reality for many businesses relying on outdated data systems, resulting in excessive infrastructure costs and a loss of competitive edge. Sida Shen from CelerData reveals how Apache Pulsar and StarRocks can transform customer-facing analytics by enabling real-time, sub-second query performance and simplifying data pipelines, illustrated through successes at companies like Airbnb and Demandbase.
What You'll Learn (Key Takeaways)
- Overcoming Data Challenges – Learn how Apache Pulsar and StarRocks address common hurdles in customer-facing analytics, such as slow query performance and complex denormalization pipelines.
- Real-Time Processing Made Simple – Discover how these technologies facilitate real-time data ingestion and processing, eliminating the need for cumbersome pre-computation.
- Optimizing Costs and Performance – Understand how integrating open formats and lakehouses can reduce storage costs and enhance data governance, paving the way for future AI applications.
Q&A Highlights
Q: Can AI agents be integrated with Apache Pulsar and StarRocks for real-time data insights?
A: Yes, an initial MCP server setup allows for building AI agents on StarRocks, enabling users to automate data interactions without custom tooling.
Q: Is there a demo available for using agents with StarRocks?
A: Yes, CelerData has a YouTube channel featuring a recent webinar on building agents, which provides a practical demonstration.
Q: Are there GitHub resources available for examples on using agents with CelerData?
A: Not currently, but plans are underway to release such resources, with the YouTube video being a recommended starting point.
Sida Shen is a contributor to the StarRocks project and a product manager at CelerData. As an engineer with a background in building machine learning and big data infrastructures, he oversees the company’s market research while working closely with engineers and developers across the analytics industry to tackle challenges related to data lakehouse analytics.
Newsletter
Our strategies and tactics delivered right to your inbox