52:44

TiDB + Pulsar, Event Streaming Architecture in Action

For distributed databases such as TiDB, Change Data Capture (CDC) is the basis for geo-distribution, database replication, backup, point in time recovery, and event-driven computation. By integrating CDC with Pulsar, we can make the critical changes in the database immediately available for processing, thus delivering insights and business values in real time.

But the implementation faces unique challenges:

Transactional ordering of events.Low latency to support database replication and event-driven processing.
Scalable to handle the increasing number of transactions per second.
Highly available and tolerant to failures.

In this talk, we will dive deep into the CDC feature in TiDB, a distributed HTAP database, and how we design, build, and integrate CDC with Pulsar.

Liquan Pei

Liquan is currently a Senior Database Engineer at PingCAP. Previously, he was the tech lead for the stream processing and real-time system on the Ads Infrastructure Team at Pinterest. Prior to that, he worked at Confluent, mainly working on the Kafka Connect framework and the HDFS and Elasticsearch connector. He is also an open-source contributor to Apache Kafka and Spark. Liquan was a speaker at Kafka Summit in 2018 and 2019.