Has anyone deploy Openmetadata successfully using AWS MWAA?

[liujuanchi]

AWS MWAA use OOS to login the managed ariflow, has no idea how to set the following parameters, can anyone help?

PIPELINE_SERVICE_CLIENT_ENDPOINT="<INGESTION_ENDPOINT_URL_WITH_SCHEME>"
SERVER_HOST_API_URL="<OPENMETADATA_ENDPOINT_URL_WITH_SCHEME>/api"
AIRFLOW_USERNAME="<AIRFLOW_UI_LOGIN_USERNAME>"
AIRFLOW_PASSWORD="<AIRFLOW_UI_LOGIN_PASSWORD>"

as AWS SSO using Identity Center, can I put username and pwd in Identity Center to these two AIRFLOW_USERNAME & AIRFLOW_PASSWORD field?

And, need any change for following fields?
AIRFLOW_TIMEOUT="10"
AIRFLOW_TRUST_STORE_PATH=""
AIRFLOW_TRUST_STORE_PASSWORD=""
PIPELINE_SERVICE_CLIENT_CLASS_NAME="org.openmetadata.service.clients.pipeline.airflow.AirflowRESTClient"
PIPELINE_SERVICE_CLIENT_HEALTH_CHECK_INTERVAL="300"
PIPELINE_SERVICE_CLIENT_VERIFY_SSL="no-ssl"
PIPELINE_SERVICE_CLIENT_SSL_CERT_PATH=""
PIPELINE_SERVICE_IP_INFO_ENABLED="false"
PIPELINE_SERVICE_CLIENT_HOST_IP=""
PIPELINE_SERVICE_CLIENT_SECRETS_MANAGER_LOADER="noop"

[Prajwal214]

To configure OpenMetadata to connect to AWS MWAA using AWS SSO, you need to set the parameters correctly. However, AWS SSO does not directly provide username and password for Airflow UI login. Instead, you should use the appropriate authentication mechanism supported by AWS SSO.

1. Set the Endpoint URLs:

• PIPELINE_SERVICE_CLIENT_ENDPOINT: URL of the Airflow API endpoint.
• SERVER_HOST_API_URL: URL of the OpenMetadata API endpoint.

2. Authentication:

• AWS SSO does not provide direct username/password for Airflow. You need to configure OpenMetadata to use AWS SSO for authentication. This might involve setting up OAuth or another supported authentication method.

3. Other Parameters:

• AIRFLOW_TIMEOUT, AIRFLOW_TRUST_STORE_PATH, AIRFLOW_TRUST_STORE_PASSWORD, PIPELINE_SERVICE_CLIENT_CLASS_NAME, PIPELINE_SERVICE_CLIENT_HEALTH_CHECK_INTERVAL, PIPELINE_SERVICE_CLIENT_VERIFY_SSL, PIPELINE_SERVICE_CLIENT_SSL_CERT_PATH, PIPELINE_SERVICE_IP_INFO_ENABLED, PIPELINE_SERVICE_CLIENT_HOST_IP, PIPELINE_SERVICE_CLIENT_SECRETS_MANAGER_LOADER can be set as per your environment requirements.

https://docs.open-metadata.org/latest/deployment/ingestion/external/mwaa

[liujuanchi]

Hi there,

thank your for the replay. but things confused me is: if I add Okta to IAM as the Idps provider, it can makes access to MWAA. also I need using Okta for Openmetadata Integration access. then how the .env would be like? From the deployment guide, It's looks the following parameters only working from the Openmetadata Apps:

OpenMetadata Server Authentication Configuration

AUTHORIZER_CLASS_NAME=org.openmetadata.service.security.DefaultAuthorizer
AUTHORIZER_REQUEST_FILTER=org.openmetadata.service.security.JwtFilter
AUTHORIZER_ADMIN_PRINCIPALS=[admin] # Your name from name@domain.com
AUTHORIZER_PRINCIPAL_DOMAIN=open-metadata.org # Update with your domain

AUTHENTICATION_PROVIDER=okta
AUTHENTICATION_PUBLIC_KEYS={ISSUER_URL}/v1/keys # Update with your Issuer URL
AUTHENTICATION_AUTHORITY={ISSUER_URL} # Update with your Issuer URL
AUTHENTICATION_CLIENT_ID={CLIENT_ID - SPA APP} # Update with your Client ID
AUTHENTICATION_CALLBACK_URL=http://localhost:8585/callback