Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

FIX - Ingestion Airflow Image constraints #17296

Merged
merged 3 commits into from
Aug 6, 2024
Merged

FIX - Ingestion Airflow Image constraints #17296

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
3 commits
Select commit Hold shift + click to select a range
3611c9e
FIX - Ingestion constraints
pmbrull Aug 5, 2024
d56784c
FIX - Ingestion constraints
pmbrull Aug 5, 2024
fc6378e
FIX - Ingestion constraints
pmbrull Aug 5, 2024
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 1 addition & 1 deletion ingestion/Dockerfile
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -70,7 +70,7 @@ ENV PIP_NO_CACHE_DIR=1
ENV PIP_QUIET=1
ARG RI_VERSION="1.5.0.0.dev0"
RUN pip install --upgrade pip
RUN pip install "openmetadata-managed-apis~=${RI_VERSION}" --constraint "https://raw.githubusercontent.com/apache/airflow/constraints-2.7.3/constraints-3.10.txt"
RUN pip install "openmetadata-managed-apis~=${RI_VERSION}" --constraint "https://raw.githubusercontent.com/apache/airflow/constraints-2.9.1/constraints-3.10.txt"
RUN pip install "openmetadata-ingestion[${INGESTION_DEPENDENCY}]~=${RI_VERSION}"

# Temporary workaround for https://github.com/open-metadata/OpenMetadata/issues/9593
Expand Down
6 changes: 3 additions & 3 deletions ingestion/Dockerfile.ci
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -65,7 +65,7 @@ COPY --chown=airflow:0 openmetadata-airflow-apis /home/airflow/openmetadata-airf
COPY --chown=airflow:0 ingestion/examples/airflow/dags /opt/airflow/dags

USER airflow
ARG AIRFLOW_CONSTRAINTS_LOCATION="https://raw.githubusercontent.com/apache/airflow/constraints-2.7.3/constraints-3.10.txt"
ARG AIRFLOW_CONSTRAINTS_LOCATION="https://raw.githubusercontent.com/apache/airflow/constraints-2.9.1/constraints-3.10.txt"

# Disable pip cache dir
# https://pip.pypa.io/en/stable/topics/caching/#avoiding-caching
Expand All @@ -76,13 +76,13 @@ ENV PIP_QUIET=1
RUN pip install --upgrade pip

WORKDIR /home/airflow/openmetadata-airflow-apis
RUN PIP_CONSTRAINT=/home/airflow/openmetadata-airflow-apis/constraints.txt pip install "."
RUN pip install "." --constraint "${AIRFLOW_CONSTRAINTS_LOCATION}"

WORKDIR /home/airflow/ingestion

# Argument to provide for Ingestion Dependencies to install. Defaults to all
ARG INGESTION_DEPENDENCY="all"
RUN PIP_CONSTRAINT=/home/airflow/ingestion/constraints.txt pip install ".[${INGESTION_DEPENDENCY}]"
RUN pip install ".[${INGESTION_DEPENDENCY}]"

# Temporary workaround for https://github.com/open-metadata/OpenMetadata/issues/9593
RUN echo "Image built for $(uname -m)"
Expand Down
10 changes: 5 additions & 5 deletions ingestion/Makefile
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -11,23 +11,23 @@ endif

.PHONY: install
install: ## Install the ingestion module to the current environment
PIP_CONSTRAINT=$(INGESTION_DIR)/constraints.txt python -m pip install $(INGESTION_DIR)/
python -m pip install $(INGESTION_DIR)/

.PHONY: install_dev
install_dev: ## Install the ingestion module with dev dependencies
PIP_CONSTRAINT=$(INGESTION_DIR)/constraints.txt python -m pip install "$(INGESTION_DIR)[dev]/"
python -m pip install "$(INGESTION_DIR)[dev]/"

.PHONY: install_test
install_test: ## Install the ingestion module with test dependencies
PIP_CONSTRAINT=$(INGESTION_DIR)/constraints.txt python -m pip install "$(INGESTION_DIR)[test]/"
python -m pip install "$(INGESTION_DIR)[test]/"

.PHONY: install_all
install_all: ## Install the ingestion module with all dependencies
PIP_CONSTRAINT=$(INGESTION_DIR)/constraints.txt python -m pip install "$(INGESTION_DIR)[all]/"
python -m pip install "$(INGESTION_DIR)[all]/"

.PHONY: install_apis
install_apis: ## Install the REST APIs module to the current environment
PIP_CONSTRAINT=$(ROOT_DIR)/openmetadata-airflow-apis/constraints.txt python -m pip install $(ROOT_DIR)/openmetadata-airflow-apis/ setuptools~=70.3.0
python -m pip install $(ROOT_DIR)/openmetadata-airflow-apis/ setuptools~=70.3.0

.PHONY: lint
lint: ## Run pylint on the Python sources to analyze the codebase
Expand Down
1 change: 0 additions & 1 deletion ingestion/constraints.txt
View file
Open in desktop

This file was deleted.

1 change: 0 additions & 1 deletion openmetadata-airflow-apis/constraints.txt
View file
Open in desktop

This file was deleted.

4 changes: 2 additions & 2 deletions openmetadata-airflow-apis/pyproject.toml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -15,9 +15,9 @@ license = {file = "LICENSE"}
description = "Airflow REST APIs to create and manage DAGS"
requires-python = ">=3.8"
dependencies = [
"pendulum~=2.1.2",
"pendulum~=3.0",
"apache-airflow>=2.2.2",
"Flask>=1.1.4",
"Flask==2.2.5",
"Flask-Admin==1.6.0",
]

Expand Down
Loading