Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

GitHub Action to Compare Dependencies #29728

Open
wants to merge 38 commits into
base: master
Choose a base branch
from
Open

Conversation

shmuel44
Copy link
Contributor

@shmuel44 shmuel44 commented Jul 16, 2024

Related Issues

Related: link to the issue

Description

Added a GitHub action to check if there is a difference between dependencies of the native Docker image and all the other images that are supported by the native image.

image

@xsoar-bot
Copy link

Docker Image Ready - Dev

Docker automatic build at CircleCI has deployed your docker image: devdemisto/py3-native:8.6.0.103670
It is available now on docker hub at: https://hub.docker.com/r/devdemisto/py3-native/tags
Get started by pulling the image:

docker pull devdemisto/py3-native:8.6.0.103670

Docker Metadata

  • Image Size: 1305.09 MB
  • Image ID: sha256:9fafc5e68541dcf47b3b0a1533ca1646a2857272c835decfad1fe1d7dbc6653b
  • Created: 2024-07-16T15:45:22.234863226Z
  • Arch: linux/amd64
  • Command: ["python3"]
  • Environment:
    • PATH=/usr/local/bin:/home/python/.local/bin:/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin
    • container=oci
    • PYTHON_VERSION=3.11.9
    • LANG=C.UTF-8
    • DOCKER_IMAGE=devdemisto/py3-native:8.6.0.103670
    • NLTK_DATA=/etc/ml/nltk_data
  • Labels:
    • architecture:x86_64
    • build-date:2024-02-29T13:57:42
    • com.redhat.component:ubi9-container
    • com.redhat.license_terms:https://www.redhat.com/en/about/red-hat-end-user-license-agreements#UBI
    • description:The Universal Base Image is designed and engineered to be the base layer for all of your containerized applications, middleware and utilities. This base image is freely redistributable, but Red Hat only supports Red Hat technologies through subscriptions for Red Hat products. This image is maintained by Red Hat and updated regularly.
    • distribution-scope:public
    • io.buildah.version:1.29.0
    • io.k8s.description:The Universal Base Image is designed and engineered to be the base layer for all of your containerized applications, middleware and utilities. This base image is freely redistributable, but Red Hat only supports Red Hat technologies through subscriptions for Red Hat products. This image is maintained by Red Hat and updated regularly.
    • io.k8s.display-name:Red Hat Universal Base Image 9
    • io.openshift.expose-services:
    • io.openshift.tags:base rhel9
    • maintainer:Red Hat, Inc.
    • name:ubi9
    • org.opencontainers.image.authors:Demisto <containers@demisto.com>
    • org.opencontainers.image.revision:9e87de4b9a5674ffaa8a2e3aac2f9af9f5a5d494
    • org.opencontainers.image.version:8.6.0.103670
    • release:1610
    • summary:Provides the latest release of Red Hat Universal Base Image 9.
    • url:https://access.redhat.com/containers/#/registry.access.redhat.com/ubi9/images/9.3-1610
    • vcs-ref:eb726081eeafc660c182aae53074ec6631cb473e
    • vcs-type:git
    • vendor:Red Hat, Inc.
    • version:9.3

@shmuel44 shmuel44 changed the title added gitub action GitHub Action to Compare Dependencies Jul 16, 2024
@shmuel44 shmuel44 self-assigned this Jul 16, 2024
@xsoar-bot
Copy link

Docker Image Ready - Dev

Docker automatic build at CircleCI has deployed your docker image: devdemisto/py3-native:8.6.0.103685
It is available now on docker hub at: https://hub.docker.com/r/devdemisto/py3-native/tags
Get started by pulling the image:

docker pull devdemisto/py3-native:8.6.0.103685

Docker Metadata

  • Image Size: 1305.10 MB
  • Image ID: sha256:f6eee534f4fe3b60f9172ad311b3d14ef7d01ae06a4e6e9960c236cf3aaee1cd
  • Created: 2024-07-16T20:12:36.022466672Z
  • Arch: linux/amd64
  • Command: ["python3"]
  • Environment:
    • PATH=/usr/local/bin:/home/python/.local/bin:/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin
    • container=oci
    • PYTHON_VERSION=3.11.9
    • LANG=C.UTF-8
    • DOCKER_IMAGE=devdemisto/py3-native:8.6.0.103685
    • NLTK_DATA=/etc/ml/nltk_data
  • Labels:
    • architecture:x86_64
    • build-date:2024-02-29T13:57:42
    • com.redhat.component:ubi9-container
    • com.redhat.license_terms:https://www.redhat.com/en/about/red-hat-end-user-license-agreements#UBI
    • description:The Universal Base Image is designed and engineered to be the base layer for all of your containerized applications, middleware and utilities. This base image is freely redistributable, but Red Hat only supports Red Hat technologies through subscriptions for Red Hat products. This image is maintained by Red Hat and updated regularly.
    • distribution-scope:public
    • io.buildah.version:1.29.0
    • io.k8s.description:The Universal Base Image is designed and engineered to be the base layer for all of your containerized applications, middleware and utilities. This base image is freely redistributable, but Red Hat only supports Red Hat technologies through subscriptions for Red Hat products. This image is maintained by Red Hat and updated regularly.
    • io.k8s.display-name:Red Hat Universal Base Image 9
    • io.openshift.expose-services:
    • io.openshift.tags:base rhel9
    • maintainer:Red Hat, Inc.
    • name:ubi9
    • org.opencontainers.image.authors:Demisto <containers@demisto.com>
    • org.opencontainers.image.revision:ed62dfb966a5d9e18355a107db386effafc56d22
    • org.opencontainers.image.version:8.6.0.103685
    • release:1610
    • summary:Provides the latest release of Red Hat Universal Base Image 9.
    • url:https://access.redhat.com/containers/#/registry.access.redhat.com/ubi9/images/9.3-1610
    • vcs-ref:eb726081eeafc660c182aae53074ec6631cb473e
    • vcs-type:git
    • vendor:Red Hat, Inc.
    • version:9.3

@shmuel44 shmuel44 requested review from dorschw and removed request for dorschw July 17, 2024 07:16
Copy link
Contributor

@dorschw dorschw left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Good job!

.github/workflows/sync_native_dependency.yml Outdated Show resolved Hide resolved
.github/workflows/sync_native_dependency.yml Outdated Show resolved Hide resolved
.github/workflows/sync_native_dependency.yml Outdated Show resolved Hide resolved
.github/workflows/sync_native_dependency.yml Outdated Show resolved Hide resolved
utils/compare_dependency_constraints.py Outdated Show resolved Hide resolved
utils/compare_dependency_constraints.py Outdated Show resolved Hide resolved
utils/compare_dependency_constraints.py Outdated Show resolved Hide resolved
utils/compare_dependency_constraints.py Outdated Show resolved Hide resolved
utils/compare_dependency_constraints.py Outdated Show resolved Hide resolved
Co-authored-by: dorschw <81086590+dorschw@users.noreply.github.com>
@xsoar-bot
Copy link

Docker Image Ready - Dev

Docker automatic build at CircleCI has deployed your docker image: devdemisto/py3-native:8.6.0.103850
It is available now on docker hub at: https://hub.docker.com/r/devdemisto/py3-native/tags
Get started by pulling the image:

docker pull devdemisto/py3-native:8.6.0.103850

Docker Metadata

  • Image Size: 1307.45 MB
  • Image ID: sha256:cc2810d373ce701dd7982be5f02c304f0f72ac1d5de9a2604b1aecb4bda46391
  • Created: 2024-07-18T11:31:48.801274247Z
  • Arch: linux/amd64
  • Command: ["python3"]
  • Environment:
    • PATH=/usr/local/bin:/home/python/.local/bin:/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin
    • container=oci
    • PYTHON_VERSION=3.11.9
    • LANG=C.UTF-8
    • DOCKER_IMAGE=devdemisto/py3-native:8.6.0.103850
    • NLTK_DATA=/etc/ml/nltk_data
  • Labels:
    • architecture:x86_64
    • build-date:2024-02-29T13:57:42
    • com.redhat.component:ubi9-container
    • com.redhat.license_terms:https://www.redhat.com/en/about/red-hat-end-user-license-agreements#UBI
    • description:The Universal Base Image is designed and engineered to be the base layer for all of your containerized applications, middleware and utilities. This base image is freely redistributable, but Red Hat only supports Red Hat technologies through subscriptions for Red Hat products. This image is maintained by Red Hat and updated regularly.
    • distribution-scope:public
    • io.buildah.version:1.29.0
    • io.k8s.description:The Universal Base Image is designed and engineered to be the base layer for all of your containerized applications, middleware and utilities. This base image is freely redistributable, but Red Hat only supports Red Hat technologies through subscriptions for Red Hat products. This image is maintained by Red Hat and updated regularly.
    • io.k8s.display-name:Red Hat Universal Base Image 9
    • io.openshift.expose-services:
    • io.openshift.tags:base rhel9
    • maintainer:Red Hat, Inc.
    • name:ubi9
    • org.opencontainers.image.authors:Demisto <containers@demisto.com>
    • org.opencontainers.image.revision:70f60399e4f05c439be2a435ac9e5e1267c04bb2
    • org.opencontainers.image.version:8.6.0.103850
    • release:1610
    • summary:Provides the latest release of Red Hat Universal Base Image 9.
    • url:https://access.redhat.com/containers/#/registry.access.redhat.com/ubi9/images/9.3-1610
    • vcs-ref:eb726081eeafc660c182aae53074ec6631cb473e
    • vcs-type:git
    • vendor:Red Hat, Inc.
    • version:9.3

@xsoar-bot
Copy link

Docker Image Ready - Dev

Docker automatic build at CircleCI has deployed your docker image: devdemisto/py3-native:8.6.0.103856
It is available now on docker hub at: https://hub.docker.com/r/devdemisto/py3-native/tags
Get started by pulling the image:

docker pull devdemisto/py3-native:8.6.0.103856

Docker Metadata

  • Image Size: 1307.44 MB
  • Image ID: sha256:511bf353ac380b92bc0ed1f50e73f260cb4bed3dbb7d00a072cca98260f9d70e
  • Created: 2024-07-18T12:19:26.721349527Z
  • Arch: linux/amd64
  • Command: ["python3"]
  • Environment:
    • PATH=/usr/local/bin:/home/python/.local/bin:/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin
    • container=oci
    • PYTHON_VERSION=3.11.9
    • LANG=C.UTF-8
    • DOCKER_IMAGE=devdemisto/py3-native:8.6.0.103856
    • NLTK_DATA=/etc/ml/nltk_data
  • Labels:
    • architecture:x86_64
    • build-date:2024-02-29T13:57:42
    • com.redhat.component:ubi9-container
    • com.redhat.license_terms:https://www.redhat.com/en/about/red-hat-end-user-license-agreements#UBI
    • description:The Universal Base Image is designed and engineered to be the base layer for all of your containerized applications, middleware and utilities. This base image is freely redistributable, but Red Hat only supports Red Hat technologies through subscriptions for Red Hat products. This image is maintained by Red Hat and updated regularly.
    • distribution-scope:public
    • io.buildah.version:1.29.0
    • io.k8s.description:The Universal Base Image is designed and engineered to be the base layer for all of your containerized applications, middleware and utilities. This base image is freely redistributable, but Red Hat only supports Red Hat technologies through subscriptions for Red Hat products. This image is maintained by Red Hat and updated regularly.
    • io.k8s.display-name:Red Hat Universal Base Image 9
    • io.openshift.expose-services:
    • io.openshift.tags:base rhel9
    • maintainer:Red Hat, Inc.
    • name:ubi9
    • org.opencontainers.image.authors:Demisto <containers@demisto.com>
    • org.opencontainers.image.revision:5268bcd5e468ed0d734c316b98a02432126e32c1
    • org.opencontainers.image.version:8.6.0.103856
    • release:1610
    • summary:Provides the latest release of Red Hat Universal Base Image 9.
    • url:https://access.redhat.com/containers/#/registry.access.redhat.com/ubi9/images/9.3-1610
    • vcs-ref:eb726081eeafc660c182aae53074ec6631cb473e
    • vcs-type:git
    • vendor:Red Hat, Inc.
    • version:9.3

@shmuel44 shmuel44 marked this pull request as ready for review July 31, 2024 13:33
Copy link
Collaborator

@JudahSchwartz JudahSchwartz left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

looks good, we should add a similiar check that python3-tools and python3-tools-ubi support each other

docker/py3-native/pyproject.toml Outdated Show resolved Hide resolved
@xsoar-bot
Copy link

Docker Image Ready - Dev

Docker automatic build at CircleCI has deployed your docker image: devdemisto/py3-native:8.7.0.108164
It is available now on docker hub at: https://hub.docker.com/r/devdemisto/py3-native/tags
Get started by pulling the image:

docker pull devdemisto/py3-native:8.7.0.108164

Docker Metadata

  • Image Size: 1256.66 MB
  • Image ID: sha256:e10d972e71bbbe09b1489f655060f2f86697dba0db52aa98fa212e5d8614ead2
  • Created: 2024-08-12T12:59:53.58836669Z
  • Arch: linux/amd64
  • Command: ["python3"]
  • Environment:
    • PATH=/usr/local/bin:/home/python/.local/bin:/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin
    • container=oci
    • PYTHON_VERSION=3.11.9
    • LANG=C.UTF-8
    • DOCKER_IMAGE=devdemisto/py3-native:8.7.0.108164
    • NLTK_DATA=/etc/ml/nltk_data
  • Labels:
    • architecture:x86_64
    • build-date:2024-07-18T15:47:19
    • com.redhat.component:ubi9-container
    • com.redhat.license_terms:https://www.redhat.com/en/about/red-hat-end-user-license-agreements#UBI
    • description:The Universal Base Image is designed and engineered to be the base layer for all of your containerized applications, middleware and utilities. This base image is freely redistributable, but Red Hat only supports Red Hat technologies through subscriptions for Red Hat products. This image is maintained by Red Hat and updated regularly.
    • distribution-scope:public
    • io.buildah.version:1.29.0
    • io.k8s.description:The Universal Base Image is designed and engineered to be the base layer for all of your containerized applications, middleware and utilities. This base image is freely redistributable, but Red Hat only supports Red Hat technologies through subscriptions for Red Hat products. This image is maintained by Red Hat and updated regularly.
    • io.k8s.display-name:Red Hat Universal Base Image 9
    • io.openshift.expose-services:
    • io.openshift.tags:base rhel9
    • maintainer:Red Hat, Inc.
    • name:ubi9
    • org.opencontainers.image.authors:Demisto <containers@demisto.com>
    • org.opencontainers.image.revision:3b5110ec1970cf8446131c769a54e34a7c616833
    • org.opencontainers.image.version:8.7.0.108164
    • release:1181
    • summary:Provides the latest release of Red Hat Universal Base Image 9.
    • url:https://access.redhat.com/containers/#/registry.access.redhat.com/ubi9/images/9.4-1181
    • vcs-ref:e309397d02fc53f7fa99db1371b8700eb49f268f
    • vcs-type:git
    • vendor:Red Hat, Inc.
    • version:9.4

def __str__(self) -> str:
return (
f"{self.dependency} is {self.in_image or 'missing'} in {self.image}, "
f"but {self.in_reference or 'missing'} in the {self.reference_image} image. "
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
f"but {self.in_reference or 'missing'} in the {self.reference_image} image. "
f"but {self.in_reference or 'missing'} in {self.reference_image}. "

Comment on lines +60 to +63
if path.suffix == PIPFILE:
return lower_dict_keys(_parse_pipfile(path))

return lower_dict_keys(_parse_pyproject(path))
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

best to have an

elif : ... 
else: raise RuntimeError

for safety

Comment on lines +102 to +111
"""Compares the dependency constraints between different images and reports discrepancies.

This function compares the dependencies of the following images:
- `py3-tools`
- `py3-tools-ubi`
- `native`

against the dependencies of the images listed in `images_contained_in_native`.

Additionally, it compares the dependencies of `py3-tools` against `py3-tools-ubi`.
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

split the docs between the two methods, this one shouldn't document the others

for dependency in sorted(
constraint_keys.difference(native_constraint_keys)
)
discrepancies.extend(compare_with_native(image, native_constraints))
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

this should be in compare_with_native

return int(bool(discrepancies))


def compare_with_native(image: str, native_constraints: dict) -> list[Discrepancy]:
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

rename to find_native_discrepancies

return discrepancies


def compare_py3_tools_with_ubi(
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
def compare_py3_tools_with_ubi(
def find_ubi_discrepancies(

Copy link

stale bot commented Sep 5, 2024

This pull request has been automatically marked as stale because it has not had recent activity. It will be closed in 7 days if no further activity occurs.

If you'd like to keep it open, please leave a comment with the status of the PR.

Thank you for your contribution!

@stale stale bot added the stale label Sep 5, 2024
Copy link

stale bot commented Sep 19, 2024

This pull request has been automatically closed because there has been no activity for 21 days.

Please feel free to reopen it (or open a new one) if the proposed change is still appropriate.

Thank you for your contribution!

@stale stale bot closed this Sep 19, 2024
@shmuel44 shmuel44 reopened this Sep 24, 2024
@stale stale bot removed the stale label Sep 24, 2024
@JudahSchwartz JudahSchwartz removed their request for review September 30, 2024 12:48
Copy link

stale bot commented Oct 29, 2024

This pull request has been automatically marked as stale because it has not had recent activity. It will be closed in 7 days if no further activity occurs.

If you'd like to keep it open, please leave a comment with the status of the PR.

Thank you for your contribution!

@stale stale bot added the stale label Oct 29, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Projects
None yet
Development

Successfully merging this pull request may close these issues.

4 participants