Dynamically define dlq and retry topics (GSI-1175) #143
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
akafka/config.py
kafka_dlq_topic
andkafka_retry_topic
). The former is generated by appendingKafkaConfig.service_name
and-dlq
to the end of the vanilla topic name, with a.
between. Example: regular topic is 'some-topic', the service name is 'my-service', and the dlq topic that 'my-service' uses for erroring events from 'some-topic' is 'some-topic.my-service-dlq'.akafka/eventsub.py
service_name
.original_topic
header for the first half of the DLQ flow (it's not needed or included when publishing a failed event to the DLQ because the topic name is preserved in the DLQ topic's name)KafkaEventSubscriber._extract_info()
, it will get the original topic value from the DLQ topic name instead of the old headerKafkaDLQSubscriber
, The retry topic is no longer an init variable because that information is, again, pulled from the service name (which is embedded in the DLQ topic name).Tests
event_subscriber
ordlq_subscriber
for claritydlq
orretry
, but these are nowtest-topic.test-dlq
ortest-retry
)