Session Overview
Discover how Netflix solves ML asset fragmentation and operational friction by building a centralized Model Lifecycle Graph.
In this talk, Saish Sali (Senior Software Engineer on Netflix's AI Platform team) breaks down the architecture and engineering principles behind Netflix’s Metadata Service (MDS). Learn how Netflix connects disparate machine learning assets—from models, features, and datasets to training pipelines and A/B test cells—into a unified, navigable graph powered by Datomic, Elasticsearch, and event streaming. Plus, see how this connected knowledge graph provides the backbone for AI agents using Model Context Protocol (MCP) to automate lineage tracing and asset discovery.
What You'll Learn
- The cost of ML silo fragmentation across business domains
- Designing a shared data vocabulary: Components, entities, domains, and global URIs
- Event-driven metadata ingestion with thin events and Kafka
- Dual-storage architecture: Datomic for graph traversals and Elasticsearch for search
- Tracing model lineage, upstream data dependencies, and downstream impact
- Enabling autonomous AI agents to explore metadata and relationships via MCP
Chapters
- 00:00 — Introduction & the ML fragmentation problem
- 02:08 — The spread of AI/ML across Netflix domains
- 04:13 — Key challenges: Discovery, lineage, and impact analysis
- 06:05 — The solution: The Model Lifecycle Graph & Metadata Service
- 07:47 — Shared data vocabulary & universal URIs
- 09:43 — Entity relationships & the connected graph
- 12:10 — Ingestion architecture: Event mesh, Kafka, Datomic & Elasticsearch
- 14:52 — Design principles: Thin events & dual storage roles
- 17:34 — Investigation walkthrough & powering AI agents via MCP
- 20:23 — Current challenges & future outlook
