Discover how real-time, in-session recommendation systems are revolutionizing user experiences at massive scale. This talk explains how we delivered personalized product recommendations with sub-second latency to over 100 million users by capturing and analyzing user behavior as it happens.
Traditional recommendation systems rely heavily on historical data, which often fails to reflect users’ immediate intent. Our approach focuses on current user interactions — clicks, dwell time, scroll patterns, and more — to predict what users want before they leave the page.
Key Components of the System:
- Client-Side Tracking Module: Lightweight JavaScript library capturing clicks, page views, scroll depth, product impressions, and cross-device session continuity while preserving user privacy.
- Real-Time Processing Pipeline: Kafka-powered streaming backbone enabling sub-second data ingestion, feature engineering, and dynamic transformation of raw behavioral data.
- Adaptive Recommendation Engine: Session-based intent modeling, real-time ranking and scoring, compliance filtering, and personalization tuning based on immediate user feedback.
Impact & Results:
- 27% increase in conversion rates
- 32% reduction in session abandonment
- Sub-second latency from action to recommendation
- 4.8× improvement in recommendation relevance compared to historical-only approaches
Challenges Overcome:
- Scaling to 100M users with billions of events daily
- Maintaining sub-second latency across all components
- Preserving privacy through edge processing and differential privacy techniques
- Ensuring resilient Kafka infrastructure for high-traffic spikes
Key Learnings:
- Session context often outweighs historical preferences
- Fresh data is critical for relevance
- Stream processing at scale requires careful architecture and tuning
- Privacy and personalization can coexist
This talk is essential for data engineers, AI practitioners, and product leaders interested in building real-time, scalable, and privacy-conscious recommendation systems. Learn how streaming architectures and adaptive AI models can deliver next-generation user experiences at massive scale.
As the Engineering Manager with over 15 years of experience overseeing the Online Data Platform, Data Frameworks, and Behavioral Tracking Platform teams , I lead strategic initiatives that build scalable, efficient data systems powering AI and machine learning-driven recommendation platforms. My hands-on technical expertise and strong leadership skills have enabled me to seamlessly transition from staff-level engineering to management, consistently driving innovation and delivering impactful, high-value solutions.I have a proven track record in designing high-throughput, low-latency systems using Kafka, Akka Streams, and Spark, which power real-time, mission-critical AI and machine learning applications. My expertise extends across the full lifecycle of data pipelines that fuel machine learning models, including Kafka, Google Dataflow, Spark Streaming, Akka Streams, BigTable, and BigQuery. Additionally, I have designed and developed ETL systems (PHP, Go, Scala) to streamline data ingestion for advanced analytics and AI algorithms.In the realm of AI, I’ve contributed to the architecture and deployment of machine learning systems that enhance Credit Karma's recommendation engines and data-driven products. By optimizing data platforms for AI inference pipelines, I ensure scalable, low-latency access to the high-quality data needed for machine learning models.
Samip Singhal heads Online Recommendation System at Intuit Credit Karma. His team owns the retrieval and ranking systems behind the app’s personalized offers, combining real-time features, rigorous experimentation, and safety controls to improve relevance and outcomes. With 15+ years building and leading recommendation engineering across marketing, retail, and fintech, Samip previously served as a Center-of-Excellence Staff Data Architect advising 30 Fortune 100 companies.His current focus is on Online Recommendation architectures that learn within a session.
Recommended resources
Watch more events.
Newsletter
Our strategies and tactics delivered right to your inbox