This guide outlines how to override the default AWS connections backend in Apache Airflow with a custom backend. This enables Airflow to access secrets stored in other AWS accounts, in addition to the current one, leveraging AWS Secrets Manager.
By default, Airflow uses the airflow.secrets
base class
for secrets backends.
To access secrets across different AWS accounts, we implemented a custom backend that extends this functionality.
In short, we extended the SecretsManagerBackend class with cross-account and cross-region secrets access.