-
Notifications
You must be signed in to change notification settings - Fork 0
Commit
This commit does not belong to any branch on this repository, and may belong to a fork outside of the repository.
- Loading branch information
1 parent
1716234
commit c589294
Showing
1 changed file
with
25 additions
and
0 deletions.
There are no files selected for viewing
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -1 +1,26 @@ | ||
# fdk-kafka-event-publisher | ||
|
||
This is deployed as 6 different services: | ||
- fdk-concept-event-publisher | ||
- fdk-data-service-event-publisher | ||
- fdk-dataset-event-publisher | ||
- fdk-event-event-publisher | ||
- fdk-information-model-event-publisher | ||
- fdk-service-event-publisher | ||
|
||
Each service connects the rabbit messages with harvest reports published by the harvesters to kafka. The harvest reports contain a list of changed resources and a list of removed resources, these are converted to kafka events. | ||
|
||
Each item in the list of removed resources will result in a remove-event and each item in the list of changed resources will result in a harvest-event. | ||
|
||
The produced kafka events: | ||
- `CONCEPT_HARVESTED` & `CONCEPT_REMOVED` | ||
- `DATA_SERVICE_HARVESTED` & `DATA_SERVICE_REMOVED` | ||
- `DATASET_HARVESTED` & `DATASET_REMOVED` | ||
- `EVENT_HARVESTED` & `EVENT_REMOVED` | ||
- `INFORMATION_MODEL_HARVESTED` & `INFORMATION_MODEL_REMOVED` | ||
- `SERVICE_HARVESTED` & `SERVICE_REMOVED` | ||
|
||
Each event contain these parameters: | ||
- `fdkId` The fdkId of the resource | ||
- `timestamp` The timestamp when the harvest started, parsed from the `startTime` field in the harvest report | ||
- `graph` The harvested graph of the resource, downloaded from the harvester for harvest events and is empty for remove events |