On Track with Apache Kafka: Building a Streaming ETL Solution with Rail Data

On Track with Apache Kafka: Building a Streaming ETL Solution with Rail Data

Register Now

Monday, Dec. 9, 2019

10:00am PST / 1:00pm ET / 6:00pm BST

As data engineers, we frequently need to build scalable systems working with data from a variety of sources and with various ingest rates, sizes, and formats. This talk takes an in-depth look at how Apache Kafka can be used to provide a common platform on which to build data infrastructure driving both real-time analytics as well as event-driven applications.

Using a public feed of railway data it will show how to ingest data from message queues such as ActiveMQ with Kafka Connect, as well as from static sources such as S3 and REST endpoints. We'll then see how to use stream processing to transform the data into a form useful for streaming to analytics in tools such as Elasticsearch and Neo4j. The same data will be used to drive a real-time notifications service through Telegram.

If you're wondering how to build your next scalable data platform, how to reconcile the impedance mismatch between stream and batch, and how to wrangle streams of data—this talk is for you!

Speakers

Robin Moffatt

Robin Moffatt, Developer Advocate, Confluent

Robin is a Developer Advocate at Confluent, the company founded by the original creators of Apache Kafka, as well as an Oracle Groundbreaker Ambassador. His career has always involved data, from the old worlds of COBOL and DB2, through the worlds of Oracle and Hadoop, and into the current world with Kafka. His particular interests are analytics, systems architecture, performance testing and optimization. He blogs at http://cnfl.io/rmoff and http://rmoff.net/ and can be found tweeting grumpy geek thoughts as @rmoff. Outside of work he enjoys drinking good beer and eating fried breakfasts, although generally not at the same time.

サイトのご利用状況を把握しユーザーエクスペリエンスの改善へとつなげるため、当サイトでは Cookie (クッキー) を使用しています。Cookie についての詳細を確認するには、また Cookie 設定の変更をご希望の場合は、こちらをクリックしてください。閲覧を続行することにより、当社の Cookie 使用に同意されたものとみなされます。