Announcing AWS SQS Connector for Apache Pulsar

Today StreamNative ships the first-ever release of the AWS Simple Queue Service (SQS) connector for Apache Pulsar. Over the last couple of months, we built an SQS connector that enables seamless integration between AWS SQS and the Pulsar ecosystem. Now we are thrilled to announce the general availability of the SQS connector! For organizations looking to deploy reactive data pipelines around Pulsar and SQS, we would recommend you try it out!

What is the SQS connector?

The SQS connector is an integration service that moves data bi-directionally between SQS and Pulsar by leveraging two types of connectors:

• SQS source connector – This connector feeds data from AWS SQS and persists data to Pulsar topics.

• SQS sink connector – This connector feeds data from Pulsar topics and persists data to AWS SQS.

Why develop the SQS connector?

Pulsar and AWS SQS are at the heart of modern cloud architectures. As one of the leading open-source distributed messaging systems, Pulsar unifies streaming and queuing capabilities and provides a broad set of features and functionalities all in one system. AWS SQS is one of the most popular queue-based message systems, offering a secure, durable, and available hosted queue that allows you to integrate and decouple distributed software systems and components. Both Pulsar and SQS have a welcoming and rapidly expanding community, and many users want to be able to leverage both the benefits of Pulsar and SQS. Moreover, we have received requests from our customers looking for an integration between Pulsar and SQS.

Why use the SQS connector?

Built to deploy integrations between Pulsar and SQS quickly and securely, the SQS connector brings various advantages, including but not limited to:

• Simplicity
• This connector simplifies integration for organizations who want to bring Pulsar into their existing infrastructure. It empowers organizations to move data in and out of Pulsar without writing a single line of code.
• Scalability
• This connector is able to run jobs on a single node (standalone) or deliver reliability at scale for an entire organization (distributed), which allows you to build reactive data pipelines to serve your business and operational needs in real-time.
• Sustainability
• Taking advantage of the full power of this connector enables you to spend less time worrying about the data layer and have more time to maximize business value from living data in an efficient manner.

Get involved

The SQS connector is a major step in the journey of integrating other message systems into the Pulsar ecosystem. To get involved with the SQS connector, check out the following featured resources:

• Try it out
• To get started, download the connector and head over to the user guides that walk you through the whole process.
• Ask a question
• Have questions? As always, feel free to create an issue on GitHub, send emails to the Pulsar mailing list, message us on Twitter, or join the #connector-sqs channel on Pulsar Slack to get answers from Pulsar experts.
• Make a contribution
• The SQS connector is a community-driven service, which hosts its source code on the StreamNative GitHub repo. We would love you to explore this new connector and contribute to its evolution. Have feature requests or bug reports? Do not hesitate to share your ideas and welcome to submit your pull request.

Today we are making the SQS connector available to you on StreamNative Hub, a centralized hub for the Pulsar ecosystem. Whether you are a longtime Pulsar user or new to Pulsar, StreamNative Hub and its rich integrations are great ways to take your organization to the next level. At StreamNative, we’re committed to the Pulsar community and will continue to invest in the Pulsar ecosystem. Today’s announcement is just another step towards realizing our vision. Stay tuned for more announcements from StreamNative!

Happy Pulsaring!