Powering Federated Learning at Tencent with Apache Pulsar

Executive Summary

Tencent Angel PowerFL is a distributed federated learning platform which can support trillions of concurrent training. Angel PowerFL has been widely used in Tencent Financial Cloud, Advertising Joint Modeling, and other businesses. The platform requires a stable and reliable messaging system with guaranteed high performance and data privacy. After investigating different solutions and comparing several messaging queues, Angel PowerFL adopted Apache Pulsar as the data synchronization solution in Federated Learning (FL).

In this blog, the Tencent Angel PowerFL team shares how they built federated communication based on Pulsar, the challenges they encountered with Pulsar, and how they solved those problems and contributed to the Pulsar community. Tencent’s use of Pulsar in production has demonstrated it provides the stability, reliability, and scalability that the machine learning platform requires.

Download and review our white paper for an in-depth look at why Federated Learning at Tencent chooses Apache Pulsar.

StreamNative is a global team of knowledgeable experts in Apache Pulsar, Apache BookKeeper, and messaging and streaming.

If you have any questions or suggestions, click here to send us an email directly. We’ll reply to you as soon as possible.

Get started on StreamNative
Free Cloud

StreamNative Free Cloud offers all of the capabilities of Pulsar without the need for a credit card.

© 北京原流科技有限公司Apache、Apache Pulsar、Apache BookKeeper、Apache Flink 及相关开源项目名称均为 Apache 软件基金会商标。条款隐私