OpenMetadata STAC Connector

This repository is an custom OpenMetadata Connector for STAC.

This repository contains a custom OpenMetadata Connector designed for extracting and ingesting STAC (SpatioTemporal Asset Catalog) metadata from STAC-compliant APIs.

The STAC Connector fetches data from your STAC catalogs, collections, and items, and structures them into OpenMetadata entities as Databases, Schemas, and Tables, making it easier for you to explore and understand your geospatial assets within the OpenMetadata ecosystem.

Step 1 - Prepare the package installation

We'll need to package the code so that it can be shipped to the ingestion container and used there. You can find a simple setup.py that builds the connector module.

Step 2 - Prepare the Ingestion Image

If you want to use the connector from the UI, the openmetadata-ingestion image should be aware of your new package.

We will be running the against the OpenMetadata version 1.5.13, therefore, our Dockerfile looks like:

FROM openmetadata/ingestion:1.5.13

# Let's use the same workdir as the ingestion image
WORKDIR /ingestion
USER airflow

# Install our custom connector
COPY connector connector
COPY setup.py .
RUN pip install requests
RUN pip install --no-deps .

Build and use the new openmetadata-ingestion images in Docker compose:

  ingestion:
    container_name: openmetadata_ingestion
    build:
      context: ../
      dockerfile: docker/Dockerfile

Step 3 - Run OpenMetadata with the custom Ingestion image

We have a Makefile prepared for you to run make run. This will get OpenMetadata up in Docker Compose using the custom Ingestion image.

You may also just run:

docker compose -f ./docker/docker-compose.yml up -d

Step 4 - Configure the Connector

In this guide we prepared a Database Connector. Thus, go to Database Services > Add New Service > Custom and set the Source Python Class Name as connector.stac_connector.STACConnector.

Note how we are specifying the full module name so that the Ingestion Framework can import the Source class.