Powering Federated Learning at Tencent with Apache Pulsar

Download and review our white paper for an in-depth look at why Federated Learning at Tencent chooses Apache Pulsar.

StreamNative is a global team of knowledgeable experts in Apache Pulsar, Apache BookKeeper, and messaging and streaming.

If you have any questions or suggestions, click here to send us an email directly. We’ll reply to you as soon as possible.

Executive Summary

Tencent Angel PowerFL is a distributed federated learning platform which can support trillions of concurrent training. Angel PowerFL has been widely used in Tencent Financial Cloud, Advertising Joint Modeling, and other businesses. The platform requires a stable and reliable messaging system with guaranteed high performance and data privacy. After investigating different solutions and comparing several messaging queues, Angel PowerFL adopted Apache Pulsar as the data synchronization solution in Federated Learning (FL).

In this blog, the Tencent Angel PowerFL team shares how they built federated communication based on Pulsar, the challenges they encountered with Pulsar, and how they solved those problems and contributed to the Pulsar community. Tencent’s use of Pulsar in production has demonstrated it provides the stability, reliability, and scalability that the machine learning platform requires.

Start streaming with Apache Pulsar on StreamNative Cloud

© StreamNative, Inc. 2021Apache, Apache Pulsar, Apache BookKeeper, Apache Flink, and associated open source project names are trademarks of the Apache Software Foundation.TermsPrivacy