Kafka Streams API is an advanced API that’s part of the Kafka Ecosystem.
Using Kafka Streams API, we can :
Apply data transformations,
Data enrichment,
Branching the data into multiple data streams.
Aggregating the data or joining data from multiple Kafka topics.
Aggregate the Data into Window Buckets and more.
The Kafka Streams API For Developers using Java/SpringBoot course is structured to give you both the theoretical and coding experience of developing Kafka Streams Applications using Streams API and also covers the techniques to use Enterprise Standard Kafka Streams Application using SpringBoot and Streams API .
This is a pure hands-on oriented course where you will be learning the concepts through code. You will build a realtime Kafka Streams application by the end of this course.
By the end of this course, you will have a complete understanding of these concepts:
Building Kafka Streams Applications using Streams API
Building Kafka Streams Applications using SpringBoot & Streams API
Write Interactive Queries to retrieve the aggregated data from a state store and expose it via RESTFUL API.
Unit and Integration Testing Kafka Streams Applications using JUnit5
Getting Started to Kafka Streams
In this section, I will give you all an introduction to Kafka streams and the different terminologies that are involved in build a Kafka Streams Application.
In this section, I will show you the best approach to build a Generic Serializer and Deserializer that can be used for any type of Messages.
Build a Generic Serializer/Deserializer
Order Management Kafka Streams application – A real time use case
In this section, we will build a kafka streams application by implementing a Order Management system for a retail company
Topology, Stream and Tasks – Under the Hood
In this section, we will explore the internals of the Kafka Streams Application.
Internals of Topology, Stream and Tasks
Error/Exception Handling in Kafka Streams
In this section, we will explore different error handlers in Kafka Streams.
Failures in Kafka Streams
Default Deserialization Error Behavior
Custom Deserialization Error Handler
Default & Custom Processor Error Handler
Custom Production Error Handler
KTable & Global KTable
In this section, we will explore the KTable and GlobalKTable in KafkaStreams.
Introduction to KTable API
Build a topology for KTable
KTable – Under the Hood
GlobalKTable
StateFul Operations in Kafka Streams – Aggregate, Join and Windowing Events
In this section, I will give an introduction to stateful operators in Kafka Streams and explore the aggregation related operators in Kafka streams.
StateFul Operations in Kafka Streams
How aggregation works ? & Aggregation using “count” operator
Group Records by using groupBy operator
Aggregation using “reduce” operartor
Aggregation using “aggregate” operator
Using Materialized views for count & reduce operator
StateFul Operation Results – How to access them ?
In this section, I will explain about the options to retrieve the results of the aggregation.
Re-Keying Kafka Records for Stateful operations
In this section, we will code and explore the effect of null operator and the need to rekeying records during stateful operations.
StateFul Operations in Kafka Streams – Join
In this section, we will code and explore the different types of Joins in Kafka Streams Application.
Join in Order Management Application – A Real Time Use Case
In this section, we will implement join in the order management application that we have been working on so far.
Introduction to Joins & Types of Joins in Kafka Streams
Explore innerJoin using “join” operator – Joining KStream and KTable
Explore innerJoin using “join” operator – Joining KStream and GlobalKTable
Explore innerJoin using “join” operator – Joining KTable and KTable
Explore innerJoin using “join” operator – Joining KStream and KStream
Joining Kafka Streams using “leftJoin” operator
Joining Kafka Streams using “outerJoin” operator
Join – Under the hood
CoPartitioning Requirements in Joins & Joins Under the Hood
StateFul Operations in Kafka Streams – Windowing
In this section, we will explore the windowing concepts in Kafka Streams.
Introduction to Windowing and time concepts
Windowing in Kafka Streams – Tumbling Windows
Control emission of windowed results using “supress” operartor
Windowing in Kafka Streams – Hopping Windows
Windowing in Kafka Streams – Sliding Windows
Widowing in Order Management Application – A Real Time Use Case
In this section, we will code and explore the new requirement to implement the windowing in the Orders Stream Application.
Behavior of Records with Future & Older Timestamp in Windowing
In this section, we will explore the behavior of records with the older and future timestamp in a Kafka Streams Application.
Build Kafka Streams Application using SpringBoot
In this section, we will build a simple kafka streams app using SpringBoot.
Introduction to SpringBoot and Kafka Streams
Setup the Project – Greeting Streams app using Spring Kafka Streams
Configuring the Kafka Stream using application.yml
Build the Greeting Topology
Test Greeting App in Local
SpringBoot AutoConfiguration of Kafka Streams
In this section, I will show you how spring boot auto configures Kafka Streams Application.
JSON Serialization/Deserialization in Spring Kafka Streams
In this section, we will implement the JSON Serialization/Deserialization in Kafka Streams using SpringBoot.
Error Handling in Spring Kafka Streams
In this section, I will show you error handling in Kafka Streams using SpringBoot.
Handle DeSerialization Error – Approach 1
Handle DeSerialization Error using Custom Error Handler – Approach 2
Handle Deserialization Error – Approach 3 ( Spring Specific Approach)
Handle UncaughtExceptions in the Topology
Handle Production Errors
Build Orders Kafka Streams Application using SpringBoot
In this section, we will set up the Spring Boot Project for orders streams.
Interactive Queries – Querying State Stores using RESTFUL APIs
Build a GET Endpoint to retrieve the OrderCount by OrderType – Part 1
Build a GET Endpoint to retrieve the OrderCount by OrderType – Part 2
Retrieve OrderCount by OrderType & LocationId
Build a GET Endpoint to retrieve the OrderCount for All OrderTypes
Build a GET Endpoint to retrieve the Revenue by OrderType
Global Error Handling for useful Client Error Messages
Interactive Queries – Querying Window State Stores using RESTFUL APIs
Build a GET Endpoint to Retrieve OrderCount by OrderType
Build a GET Endpoint to Retrieve the windowed OrderCount for All OrderTypes
Build a GET endpoint to retrieve the windowed OrderCount within a Time Range
Build a GET Endpoint to retrieve the Revenue by OrderType
Testing Kafka Streams Using TopologyTestDriver & JUnit5
In this section, we will code and learn about how to write automated tests for Kafka Streams app.
Testing Kafka Streams using TopologyTestDriver
Unit Testing Greetings App – Writing Data to a Output Topic
Unit Testing Greetings App – Testing Multiple Messages
Unit Testing Greetings App – Error Scenario
Unit Testing OrdersCount – Writing Data to a State Store
Unit Testing OrdersRevenue – Writing Data to a State Store
Unit Testing OrdersRevenue By Windows – Writing Data to a State Store
Limitations of TopologyTestDriver
Testing Kafka Streams in SpringBoot Using TopologyTestDriver & JUnit5
In this section, we will code and learn how to write unit tests in our Kafka Streams application that’s build using SpringBoot.
Integration Testing Spring KafkaStreams App using @EmbeddedKafka
In this section, we will code and learn about writing integration tests for the Kafka Streams app using EmbeddedKafka.
Introduction & SetUp Integration Test
Integration Test for OrdersCount
Integration Test for OrdersRevenue
Integration Test for OrdersRevenue By Windows
Grace Period in Kafka Streams
In this section I will explain the concept of grace period and how it can be used in kafka streams application.
Build and Package the SpringBoot App as an Executable
In this section, we will package the kafka streams app as an executable and launch the app.
By the end of this course you will have a complete understanding of Kafka Streams API and the different kinds of applications that can be built using Kafka Streams API.
Who this course is for:
Advanced Java Developers
Kafka Developers who are curious to learn Kafka Streams API
Kafka Developers who are interested in building advanced streaming applications
Developers who wish to learn the techniques to test Kafka Streams Application using TopologyTestDriver
Requirements
Java Knowledge is required
Prior experience building Kafka Applications
Prior experience working with IntelliJ or any other IDEA
Java 17 is required
Gradle or Maven Knowledge is needed
Last Updated 3/2023
VISITOR COMMENTS (0 )
FILE LIST
Filename
Size
.pad/0
63.8 KB
.pad/1
9.6 KB
.pad/10
48.5 KB
.pad/11
88.7 KB
.pad/12
176 B
.pad/13
60.8 KB
.pad/14
4 B
.pad/15
92.8 KB
.pad/16
275 B
.pad/17
112.2 KB
.pad/18
5.4 KB
.pad/19
34.3 KB
.pad/2
14.2 KB
.pad/20
674 B
.pad/21
87 KB
.pad/22
8 KB
.pad/3
1.2 KB
.pad/4
49.6 KB
.pad/5
4.5 KB
.pad/6
26.8 KB
.pad/7
11.8 KB
.pad/8
23.6 KB
.pad/9
68.4 KB
TutsNode.net.txt
63 B
[TutsNode.net] - Kafka Streams API For Developers using JavaSpringBoot/10. KTable & Global KTable/2. Build a topology for KTable.mp4
85.3 MB
[TutsNode.net] - Kafka Streams API For Developers using JavaSpringBoot/10. KTable & Global KTable/3. KTable - Under the Hood.mp4
74.8 MB
[TutsNode.net] - Kafka Streams API For Developers using JavaSpringBoot/10. KTable & Global KTable/4. Assignment - Implement the mapValues operator in KTable.html
209 B
[TutsNode.net] - Kafka Streams API For Developers using JavaSpringBoot/10. KTable & Global KTable/5.1 7.4-global-k-table.zip
193.3 KB
[TutsNode.net] - Kafka Streams API For Developers using JavaSpringBoot/11. StateFul Operations in Kafka Streams - Aggregate, Join and Windowing Events/2. What is Aggregation & How it works .mp4
94.6 MB
[TutsNode.net] - Kafka Streams API For Developers using JavaSpringBoot/11. StateFul Operations in Kafka Streams - Aggregate, Join and Windowing Events/5. Aggregation using aggregate operator.mp4
99.2 MB
[TutsNode.net] - Kafka Streams API For Developers using JavaSpringBoot/13. Aggregation in Order Management Application - A Real Time Use Case/2. Total Revenue by each store using aggregate operator.mp4
118.2 MB
[TutsNode.net] - Kafka Streams API For Developers using JavaSpringBoot/15. StateFul Operations in Kafka Streams - Join/2. Explore innerJoin using join operator - Joining KStream and KTable.mp4
119.3 MB
[TutsNode.net] - Kafka Streams API For Developers using JavaSpringBoot/15. StateFul Operations in Kafka Streams - Join/5. Explore innerJoin using join operator - Joining KStream and KStream.mp4
89.6 MB
[TutsNode.net] - Kafka Streams API For Developers using JavaSpringBoot/15. StateFul Operations in Kafka Streams - Join/8.1 11.8-under-the-hood.zip
229.2 KB
[TutsNode.net] - Kafka Streams API For Developers using JavaSpringBoot/15. StateFul Operations in Kafka Streams - Join/8.2 Kafka Commands Link.html
140 B
[TutsNode.net] - Kafka Streams API For Developers using JavaSpringBoot/15. StateFul Operations in Kafka Streams - Join/9.1 11.9-co-partioning.zip
229.4 KB
[TutsNode.net] - Kafka Streams API For Developers using JavaSpringBoot/16. Join in Order Management Application - A Real Time Use Case/1. Join Aggregate Revenue with StoreDetails KTable.mp4
94.5 MB
[TutsNode.net] - Kafka Streams API For Developers using JavaSpringBoot/16. Join in Order Management Application - A Real Time Use Case/2. Join Aggregate Count with Stores KTable.html
209 B
[TutsNode.net] - Kafka Streams API For Developers using JavaSpringBoot/17. StateFul Operations in Kafka Streams - Windowing/2. Windowing in Kafka Streams - Tumbling Windows.mp4
134.9 MB
[TutsNode.net] - Kafka Streams API For Developers using JavaSpringBoot/17. StateFul Operations in Kafka Streams - Windowing/2.1 13.2-tumbling-windows.zip
233 KB
[TutsNode.net] - Kafka Streams API For Developers using JavaSpringBoot/17. StateFul Operations in Kafka Streams - Windowing/3.1 13.3-supress.zip
233.2 KB
[TutsNode.net] - Kafka Streams API For Developers using JavaSpringBoot/17. StateFul Operations in Kafka Streams - Windowing/4.1 13.4-hopping-windows.zip
233.8 KB
[TutsNode.net] - Kafka Streams API For Developers using JavaSpringBoot/17. StateFul Operations in Kafka Streams - Windowing/5.1 13.5-sliding-windows.zip
233.8 KB
[TutsNode.net] - Kafka Streams API For Developers using JavaSpringBoot/18. Widowing in Order Management Application - A Real Time Use Case/2.1 14.2-custom-timestamp-extractor.zip
235 KB
[TutsNode.net] - Kafka Streams API For Developers using JavaSpringBoot/18. Widowing in Order Management Application - A Real Time Use Case/3. Aggregate Number of Orders by Windows.mp4
119.1 MB
[TutsNode.net] - Kafka Streams API For Developers using JavaSpringBoot/18. Widowing in Order Management Application - A Real Time Use Case/3.1 14.3-aggregate-orders-by-windows.zip
238.5 KB
[TutsNode.net] - Kafka Streams API For Developers using JavaSpringBoot/18. Widowing in Order Management Application - A Real Time Use Case/4. Aggregate Revenue by Windows.mp4
98.5 MB
[TutsNode.net] - Kafka Streams API For Developers using JavaSpringBoot/18. Widowing in Order Management Application - A Real Time Use Case/4.1 14.4-aggregate-orders-by-windows.zip
239 KB
[TutsNode.net] - Kafka Streams API For Developers using JavaSpringBoot/18. Widowing in Order Management Application - A Real Time Use Case/5.1 14.5-joins-inwindowed-data.zip
239.4 KB
[TutsNode.net] - Kafka Streams API For Developers using JavaSpringBoot/19. Behavior of Records with Future & Older Timestamp in Windowing/1.1 16.1-timestamp-behavior.zip
242 KB
[TutsNode.net] - Kafka Streams API For Developers using JavaSpringBoot/20. Build Kafka Streams Application using SpringBoot/2.1 17.2-set-up-springboot-kafka-streams.zip
103.1 KB
[TutsNode.net] - Kafka Streams API For Developers using JavaSpringBoot/22. JSON SerializationDeserialization in Spring Kafka Streams/2.1 18.2-jackson-custom-objectmapper.zip
218.9 KB
[TutsNode.net] - Kafka Streams API For Developers using JavaSpringBoot/23. Error Handling in Spring Kafka Streams/3.2 Spring for Kafka Reference Link.html
135 B
[TutsNode.net] - Kafka Streams API For Developers using JavaSpringBoot/23. Error Handling in Spring Kafka Streams/4. Handle UncaughtExceptions in the Topology.mp4
76.9 MB
[TutsNode.net] - Kafka Streams API For Developers using JavaSpringBoot/24. Build Orders Kafka Streams Application using SpringBoot/2. Create the OrdersTopology.mp4
95.2 MB
[TutsNode.net] - Kafka Streams API For Developers using JavaSpringBoot/24. Build Orders Kafka Streams Application using SpringBoot/2.1 20.2-orders-streams-setup-topology.zip
353 KB
[TutsNode.net] - Kafka Streams API For Developers using JavaSpringBoot/25. Interactive Queries - Querying State Stores using RESTFUL APIs/1. Build a GET Endpoint to retrieve the OrderCount by OrderType - Part 1.mp4
109.7 MB
[TutsNode.net] - Kafka Streams API For Developers using JavaSpringBoot/25. Interactive Queries - Querying State Stores using RESTFUL APIs/1.1 21.1-get-endpoint-orderCount-part1.zip
353 KB
[TutsNode.net] - Kafka Streams API For Developers using JavaSpringBoot/25. Interactive Queries - Querying State Stores using RESTFUL APIs/2. Build a GET Endpoint to retrieve the OrderCount by OrderType - Part 2.mp4
88.4 MB
[TutsNode.net] - Kafka Streams API For Developers using JavaSpringBoot/25. Interactive Queries - Querying State Stores using RESTFUL APIs/2.1 21.1-get-endpoint-orderCount-part2.zip
358.2 KB
[TutsNode.net] - Kafka Streams API For Developers using JavaSpringBoot/25. Interactive Queries - Querying State Stores using RESTFUL APIs/3.1 21.2-get-endpoint-orderCount.zip
358.7 KB
[TutsNode.net] - Kafka Streams API For Developers using JavaSpringBoot/25. Interactive Queries - Querying State Stores using RESTFUL APIs/4.1 21.3-get-endpoint-all-ordercount.zip
359.7 KB
[TutsNode.net] - Kafka Streams API For Developers using JavaSpringBoot/25. Interactive Queries - Querying State Stores using RESTFUL APIs/5. Build a GET Endpoint to retrieve the Revenue by OrderType.mp4
88.4 MB
[TutsNode.net] - Kafka Streams API For Developers using JavaSpringBoot/25. Interactive Queries - Querying State Stores using RESTFUL APIs/5.1 21.4-get-endpoint-orderRevenue.zip
362 KB
[TutsNode.net] - Kafka Streams API For Developers using JavaSpringBoot/25. Interactive Queries - Querying State Stores using RESTFUL APIs/6.1 21.5-global-error-handler.zip
369.8 KB
[TutsNode.net] - Kafka Streams API For Developers using JavaSpringBoot/25. Interactive Queries - Querying State Stores using RESTFUL APIs/7. Assignment Retrieve Revenue by OrderType & LocationId.html
209 B
[TutsNode.net] - Kafka Streams API For Developers using JavaSpringBoot/26. Interactive Queries - Querying Window State Stores using RESTFUL APIs/1. Build a GET Endpoint to Retrieve OrderCount by OrderType.mp4
133.8 MB
[TutsNode.net] - Kafka Streams API For Developers using JavaSpringBoot/26. Interactive Queries - Querying Window State Stores using RESTFUL APIs/1.1 22.1-get-windows-order-count-bytype.zip
375.1 KB
[TutsNode.net] - Kafka Streams API For Developers using JavaSpringBoot/26. Interactive Queries - Querying Window State Stores using RESTFUL APIs/2.1 22.2-get-windows-order-count-all-types.zip
375.5 KB
[TutsNode.net] - Kafka Streams API For Developers using JavaSpringBoot/26. Interactive Queries - Querying Window State Stores using RESTFUL APIs/3. Build a GET endpoint to retrieve the windowed OrderCount within a Time Range.mp4
95.2 MB
[TutsNode.net] - Kafka Streams API For Developers using JavaSpringBoot/26. Interactive Queries - Querying Window State Stores using RESTFUL APIs/3.1 22.3-get-windows-order-count-within-range.zip
376.7 KB
[TutsNode.net] - Kafka Streams API For Developers using JavaSpringBoot/27. Testing Kafka Streams Using TopologyTestDriver & JUnit5/1.1 Testing Streams Code.html
145 B
[TutsNode.net] - Kafka Streams API For Developers using JavaSpringBoot/27. Testing Kafka Streams Using TopologyTestDriver & JUnit5/2.1 23.2-unit-testing-greetings-app.zip
251.9 KB
[TutsNode.net] - Kafka Streams API For Developers using JavaSpringBoot/27. Testing Kafka Streams Using TopologyTestDriver & JUnit5/3.1 23.3-unit-testing-greetings-app-multiple-messages.zip
252.4 KB
[TutsNode.net] - Kafka Streams API For Developers using JavaSpringBoot/27. Testing Kafka Streams Using TopologyTestDriver & JUnit5/4.1 23.4-unit-testing-greetings-app-error.zip
252.5 KB
[TutsNode.net] - Kafka Streams API For Developers using JavaSpringBoot/27. Testing Kafka Streams Using TopologyTestDriver & JUnit5/5.1 23.5-unit-testing-orders-count.zip
262.1 KB
[TutsNode.net] - Kafka Streams API For Developers using JavaSpringBoot/27. Testing Kafka Streams Using TopologyTestDriver & JUnit5/6.1 23.6-unit-testing-orders-Revenue.zip
262.3 KB
[TutsNode.net] - Kafka Streams API For Developers using JavaSpringBoot/27. Testing Kafka Streams Using TopologyTestDriver & JUnit5/7. Unit Testing OrdersRevenue By Windows - Writing Data to a State Store.mp4
90.4 MB
[TutsNode.net] - Kafka Streams API For Developers using JavaSpringBoot/27. Testing Kafka Streams Using TopologyTestDriver & JUnit5/7.1 23.7-unit-testing-orders-Revenue-by-windows.zip
263.8 KB
[TutsNode.net] - Kafka Streams API For Developers using JavaSpringBoot/27. Testing Kafka Streams Using TopologyTestDriver & JUnit5/8.1 23.8-topology-test-driver-limitations.zip
264.5 KB
[TutsNode.net] - Kafka Streams API For Developers using JavaSpringBoot/28. Testing Kafka Streams in SpringBoot Using TopologyTestDriver & JUnit5/1.1 24.1-topologyTestdriver-springboot.zip
249.7 KB
[TutsNode.net] - Kafka Streams API For Developers using JavaSpringBoot/29. Integration Testing Spring KafkaStreams App using @EmbeddedKafka/1.1 25.1-intg-test.zip
249.7 KB
[TutsNode.net] - Kafka Streams API For Developers using JavaSpringBoot/29. Integration Testing Spring KafkaStreams App using @EmbeddedKafka/2. Integration Test for OrdersCount.mp4
96.2 MB
[TutsNode.net] - Kafka Streams API For Developers using JavaSpringBoot/29. Integration Testing Spring KafkaStreams App using @EmbeddedKafka/2.1 25.2-intg-test-orders-count.zip
253.2 KB
[TutsNode.net] - Kafka Streams API For Developers using JavaSpringBoot/29. Integration Testing Spring KafkaStreams App using @EmbeddedKafka/3. Integration Test for OrdersRevenue.mp4
79.9 MB
[TutsNode.net] - Kafka Streams API For Developers using JavaSpringBoot/29. Integration Testing Spring KafkaStreams App using @EmbeddedKafka/3.1 25.3-intg-test-orders-revenue.zip
256 KB
[TutsNode.net] - Kafka Streams API For Developers using JavaSpringBoot/29. Integration Testing Spring KafkaStreams App using @EmbeddedKafka/4.1 25.4-intg-test-orders-revenue-by-windows.zip
256 KB
[TutsNode.net] - Kafka Streams API For Developers using JavaSpringBoot/3. Greetings Kafka Streams App using KStreams API/2.1 2.2-greetings-app-setup.zip
127.6 KB
[TutsNode.net] - Kafka Streams API For Developers using JavaSpringBoot/3. Greetings Kafka Streams App using KStreams API/2.2 Github Link.html
128 B
[TutsNode.net] - Kafka Streams API For Developers using JavaSpringBoot/30. Grace Period in Kafka Streams/1. Grace Period in Windowing.mp4
90.8 MB
[TutsNode.net] - Kafka Streams API For Developers using JavaSpringBoot/30. Grace Period in Kafka Streams/1.1 26.1-grace-period.zip
256.7 KB
[TutsNode.net] - Kafka Streams API For Developers using JavaSpringBoot/31. Build and Package the SpringBoot App as an Executable/1.1 27.1-kafka-streams-jar.zip
277.4 KB
[TutsNode.net] - Kafka Streams API For Developers using JavaSpringBoot/5. Serialization and Deserialization in Kafka Streams/4.1 4.3-integrate-custom-serdes.zip
141.4 KB
[TutsNode.net] - Kafka Streams API For Developers using JavaSpringBoot/7. Order Management Kafka Streams application - A real time use case/1.1 5.1-overview-of-retail-app.zip
146.4 KB
[TutsNode.net] - Kafka Streams API For Developers using JavaSpringBoot/9. ErrorException Handling in Kafka Streams/3.1 6.5-custom-deserialization-error-handler.zip
182.2 KB
[TutsNode.net] - Kafka Streams API For Developers using JavaSpringBoot/9. ErrorException Handling in Kafka Streams/4. Default & Custom Processor Error Handler.mp4
102.4 MB
[TutsNode.net] - Kafka Streams API For Developers using JavaSpringBoot/9. ErrorException Handling in Kafka Streams/4.1 6.6-default-custom-processor-error-handler.zip
187.1 KB
[TutsNode.net] - Kafka Streams API For Developers using JavaSpringBoot/9. ErrorException Handling in Kafka Streams/5.1 6.7-default-custom-production-handler.zip