Zilliz

StreamNative and Zilliz enable real-time vector search on streaming data. Using the Kafka Connect Milvus Sink Connector, data flows continuously from Kafka topics into Zilliz's Milvus vector database, keeping vector indexes fresh for AI applications.

Kafka Connect Milvus Sink

The official Milvus sink connector integrates with StreamNative's Kafka-compatible API to:

• Stream embeddings directly from Kafka topics into Milvus collections
• Auto-index new vectors as they arrive for immediate searchability
• Handle schema evolution as embedding models change over time

Embedding Pipelines

The integration supports end-to-end embedding pipelines that transform streaming text into vectors on the fly. As new data arrives through StreamNative, it's automatically vectorized and indexed in Milvus, ensuring AI applications always operate on the most current information.

Zilliz Academy

Zilliz offers an Academy course covering the StreamNative integration, teaching developers how to build real-time RAG systems, recommendation engines, and semantic search applications using the Kafka-to-Milvus pipeline.

Use Cases

• Semantic search — keep search indexes updated in real time as content is published or modified
• Recommendation engines — continuously update user and item embeddings for personalized recommendations
• Anomaly detection — stream operational data, vectorize patterns, and detect anomalies via similarity search