Kafka

Overview

  Apache Kafka is a real-time data pipeline processor. It high-scalability, fault tolerance, execution speed, and fluid integrations are some of the key hallmarks that make it an integral part of many Enterprise Data architectures. The Apache Kafka distributed streaming platform is one of the most powerful and widely used reliable streaming platforms. Kafka is a fault-tolerant, highly scalable and used for log aggregation, stream processing, event sources, and commit logs. Kafka is used by J.B. Hunt, LinkedIn, Yahoo, Twitter, Square, Uber, Box, PayPal, Etsy, and more to enable stream processing, online messaging, facilitate in-memory computing by providing a distributed commit log, data collection for big data and so much more.   This course focuses on programming rather than the configuration management of Kafka clusters or DevOps. It starts off with the installation and setting up the development environment, before quickly moving on to performing fundamental messaging operations such as validation and enrichment. Working in a hands-on environment, you will explore message composition with pure Kafka API and Kafka Streams as well as the transformation of messages in different formats, such as text, binary, XML, JSON, and AVRO. Next, you will learn how to expose the schemas contained in Kafka with the Schema Registry. You will then learn how to work with all relevant connectors with Kafka Connect.  

Objectives

  Throughout the course, students will be led through a series of progressively advanced topics, where each topic consists of lecture, group discussion, comprehensive hands-on lab exercises, and lab review.   Working in a hands-on learning environment, students will explore

• · Solve practical large data and processing challenges with Kafka
• · Tackle data processing challenges like late events, windowing, and watermarking
• · Understand real-time streaming applications processing using Schema Registry, Kafka connect, Kafka streams, and KSQL
• · How to validate data with Kafka
• · Add information to existing data flows
• · Generate new information through message composition
• · How to perform message Serialization and Deserialization

 

Audience & Pre-Requisites

  This is an intermediate-level,skills-based hands-on training course that requires intermediate-level developer skills and knowledge. Attendees should be experienced developers who are comfortable with Java and have reasonable experience working with databases. Students should also be able to navigate the Linux command line, and have basic knowledge of Linux editors (such as VI / nano) for editing code. Experience with messaging systems or streaming is also helpful.   Take Before: Students should have attended the course(s) below, or should have basic skills in these areas: Core Java Programming Fundamentals. Please note that students without Java background may complete the labs and cut and paste or follow along as needed with the trainer. We do not cover basic Java programming in the course.