Skip to content

Sample Apache Kafka taking a stream of data changes and caching it in the Kafka store for querying

Notifications You must be signed in to change notification settings

mammenj/kstream-ktable-sample

Folders and files

NameName
Last commit message
Last commit date

Latest commit

 

History

25 Commits
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 

Repository files navigation

What is this app?

This is an example of a Spring Cloud Stream processor using Kafka Streams support

The application uses one inputs - one KStream for User changes and groups by the User key into KTable 'allusers' then streams out the changes to 'usertable'

Running the app:

  1. Start Kafka and Zookeepr

    'cd #kafka_installed_home'
    'start bin\windows\zookeeper-server-start.bat config\zookeeper.properties'
    'start bin\windows\kafka-server-start.bat config\server.properties'
  2. Build the Spring boot application

    'mvn clean package'
  3. Run the Spring boot application

    'java -jar target\kafka-streams-global-table-0.0.1-SNAPSHOT.jar'
  4. Create a producer for 'user' topic

    'start bin\windows\kafka-console-producer.bat --broker-list localhost:9092 --topic user'
  5. Input the data

    Then input the below events one by one to add to the Ktable:
    {"id": "1", "name": "paul", "age": 44}
    {"id": "2", "name": "john", "age": 43}
    {"id": "3", "name": "virat", "age": 45}
    {"id": "4", "name": "asha", "age": 100}
    {"id": "5", "name": "paul", "age": 990}
  6. Update data:

    {"id": "2", "name": "john", "age": 99}
    {"id": "3", "name": "virat", "age": 0}
    {"id": "3", "name": "kohli"}
    {"id": "3", "age": 33}

View the data in KTable

Spring boot will also printout the table every 5 seconds on the console.

Configuration (application.yml)

' spring.application.name: stream-global-table spring.cloud.stream.bindings.process-in-0: destination: user spring.cloud.stream.bindings.process-out-0: destination: usertable spring.cloud.stream.kafka.streams.binder.serdeError: logAndContinue '

About

Sample Apache Kafka taking a stream of data changes and caching it in the Kafka store for querying

Topics

Resources

Stars

Watchers

Forks

Releases

No releases published

Packages

No packages published

Languages