- 在Spark Streaming中使用Kafka作为数据源
- 利用MySQL保存Kafka offset偏移量来记录消费的位置
- Spark程序重启后可以从上次消费位置继续消费，不会丢失数据
- 使用checkpoint保证状态结果不丢失
-
Notifications
You must be signed in to change notification settings - Fork 43
E-SoulDataGroup/spark_streaming_kafka_offset
Folders and files
| Name | Name | Last commit message | Last commit date | |
|---|---|---|---|---|
Repository files navigation
About
SparkStreaming中利用MySQL保存Kafka偏移量保证0数据丢失
Topics
Resources
Stars
Watchers
Forks
Releases
No releases published
Packages 0
No packages published