Project Metamorphosis: Unveiling the next-gen event streaming platform.Learn More
Apache Kafka への接続

How the New Confluent Hub Makes Finding Connectors Easier Than Ever

Confluent Hub launched in 2018 as a place to discover and share Apache Kafka® and Confluent Platform plugins. Users have found the site a much better place to discover useful code than seeking out individual git repos. These plugins include connectors, Single Message Transforms (SMTs), and converters for Kafka Connect. At launch, we had a humble inventory of about a dozen plugins. Now, there are over 140 available, including 100+ that are fully supported by Confluent and our partners!

Since its launch, we’ve looked for ways to improve Confluent Hub, and today we’re happy to introduce a complete frontend redesign, based on user feedback. In particular, we’ve tried to make it much easier to parse plugins across dimensions that matter most to developers.

Finding what you need

Confluent Hub

The most notable update to Confluent Hub is the addition of a detailed plugin taxonomy that lets you filter options across a number of key plugin dimensions.

Plugin type

Determining the plugin type you need is likely your first concern as a developer. The plugin types on Confluent Hub are as follows:

  • Sink connector: sink connectors consume messages from Kafka topics and export them to external systems (e.g., Amazon S3 or MySQL).
  • Source connector: source connectors consume data from external systems (e.g., Amazon S3 or MySQL) and write them to Kafka topics.
  • Transform: SMTs are applied to messages as they flow through Connect. SMTs transform inbound messages after a source connector has produced them but before they are written to Kafka. Alternatively, SMTs transform outbound messages before they are sent to a sink connector.
  • Converter: converters change the format of data from bytes to a Connect internal data format, and vice versa. They are necessary in order for a Kafka Connect deployment to support a particular data format when writing to or reading from Kafka.

Enterprise support

Depending on where you’re doing your development, you may or may not need access to enterprise-level support. You might even be quite adventurous with your own projects. Lots of companies we talk to, however, aren’t comfortable putting something into production without the ability to get expert support on the phone. To help both types of users navigate the available plugins, Confluent Hub lets you filter by enterprise support type:

  • Confluent supported: Confluent enterprise support for these plugins is included as part of a Confluent Platform subscription. (You can contact us to learn more.)
  • Partner supported: enterprise support is available from the partner who maintains the plugin.
  • None: Many open source plugins fall in this category. No enterprise support is available, although varying levels of community support may be available. An open source Git repo and the Confluent Community Slack are often good places to look.

Verification

Similar to enterprise support, you may or may not be developing in an environment that requires everything to be ready for “enterprise scale” from day zero. It’s impossible to point to a single benchmark that quantifies this, but these verification levels are our attempt to identify at least those plugins that are known to be built for high performance and stability. These categories include:

  • Confluent Built: built, tested, and supported by Confluent.
  • Confluent Tested: community plugins, which are tested and supported by Confluent.
  • Verified Gold: partner built and supported integrations that meet the highest level of functional compatibility with the Confluent Platform ecosystem. (See the Confluent Verified Integration FAQs for further details.)
  • Verified Standard: partner built and supported integrations that meet a standard level of functional compatibility with the Confluent Platform ecosystem. (See the Confluent Verified Integration FAQs for further details.)
  • None: community contributions that have not been verified or tested by Confluent. This doesn’t mean they’re not good—we just haven’t had a chance to rigorously test them.

License

This one is pretty straightforward:

  • Commercial: requires a Confluent Platform or partner subscription. Confluent supported plugins can be used for free on a single broker cluster (Developer License) or free for 30 days on an unlimited number of brokers (Evaluation License).
  • Free: available for free through an open source or community license.

Available in Confluent Cloud

Our fully managed Apache Kafka as a service product, Confluent Cloud, offers some of these connectors, fully managed, on leading public clouds like Google Cloud Platform (GCP), Microsoft Azure, and Amazon Web Services (AWS).

Let us know what you think or submit a plugin

That’s a quick overview of what’s changed in the Confluent Hub frontend. We’re always looking for ways to improve the product and serve the community better, and one of the key ways we do this is by adding new open source and partner connectors to Confluent Hub. If you would like to get your code in front of more developers, or if you’d just like to give some feedback, send us an email at confluent-hub@confluent.io. We look forward to hearing from you!

Ethan Ruhe is a product manager at Confluent, where he leads the development of the Confluent CLI and formerly the Confluent Hub.

Did you like this blog post? Share it now

Subscribe to the Confluent blog

More Articles Like This

Webify Event Streams Using the Kafka Connect HTTP Sink Connector

The goal of this post is to illustrate PUSH to web from Apache Kafka® with a hands-on example. Our business users are always wanting their data faster so they can […]

What’s New in Apache Kafka 2.5

On behalf of the Apache Kafka® community, it is my pleasure to announce the release of Apache Kafka 2.5.0. The community has created another exciting release. We are making progress […]

How to Make Your Open Source Apache Kafka Connector Available on Confluent Hub

Do you have data you need to get into or out of Apache Kafka®? Kafka connectors are perfect for this. There are many connectors out there, usually for well-known and […]

Sign Up Now

最初の3か月間は各月の料金が最大50ドル割引。

新規登録のみ。

By clicking “sign up” above you understand we will process your personal information in accordance with our プライバシーポリシー

上記の「新規登録」をクリックすることにより、お客様は以下に同意するものとします。 サービス利用規約 Confluent からのマーケティングメールの随時受信にも同意するものとします。また、当社がお客様の個人情報を以下に従い処理することを理解されたものとみなします: プライバシーポリシー

単一の Kafka Broker の場合には永遠に無料
i

商用版の機能を単一の Kafka Broker で無期限で使用できるソフトウェアです。2番目の Broker を追加すると、30日間の商用版試用期間が自動で開始します。この制限を単一の Broker へ戻すことでリセットすることはできません。

デプロイのタイプを選択
Manual Deployment
  • tar
  • zip
  • deb
  • rpm
  • docker
または
自動デプロイ
  • kubernetes
  • ansible

By clicking "download free" above you understand we will process your personal information in accordance with our プライバシーポリシー

以下の「ダウンロード」をクリックすることにより、お客様は以下に同意するものとします。 Confluent ライセンス契約 Confluent からのマーケティングメールの随時受信にも同意するものとします。また、お客様の個人データが以下に従い処理することにも同意するものとします: プライバシーポリシー

このウェブサイトでは、ユーザーエクスペリエンスの向上に加え、ウェブサイトのパフォーマンスとトラフィック分析のため、Cookie を使用しています。また、サイトの使用に関する情報をソーシャルメディア、広告、分析のパートナーと共有しています。