Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Produce requirements.txt release artifacts #3725

Open
devinrsmith opened this issue Apr 24, 2023 · 3 comments
Open

Produce requirements.txt release artifacts #3725

devinrsmith opened this issue Apr 24, 2023 · 3 comments
Assignees
Milestone

Comments

@devinrsmith
Copy link
Member

It would be good to produce a set of requirements.txt release artifacts, with the knowledge that these requirements.txt files have all been through our unit and integration testing frameworks (and potential further QA).

It is likely that some matrix of requirements.txt would need to be produced. At a minimum, it would need to be per python version, and potentially as an addition, multiplied by each "extra" desired set of packages (all-ai, nltk, pytorch, etc..).

The downstream consumers of these files would be:

This is not a trivial task.

Firstly, we don't test a matrix of python versions at the moment - only 3.10. #3724

Secondly, we have a sort of cyclical dependency between https://github.com/deephaven/deephaven-server-docker and this repository. #3723

Related #3426

@devinrsmith devinrsmith added this to the Backlog milestone Apr 24, 2023
@devinrsmith devinrsmith self-assigned this Apr 24, 2023
@devinrsmith
Copy link
Member Author

As an additional upside, this might be a nice pattern for advanced integrators who want to specify their own requirements file and ensure all of the DHC unit / integration testing works.

@devinrsmith
Copy link
Member Author

There may be architecture-specific flavors that need to be accounted for as well (amd64 v arm64).

@devinrsmith
Copy link
Member Author

There may be value in publishing the docker/server-jetty/src/main/server-jetty/requirements.txt, even though it's a bit limited in scope (only tested on amd64, only python 3.10, etc...).

It's up for debate whether we want / need to strip out the "optional" dependencies from this list (related deephaven/deephaven-server-docker#71) such as turbodbc which needs to be purpose-built. That said, deephaven-server-docker as a downstream consumer does have the logic to build that wheel as necessary.

devinrsmith added a commit to devinrsmith/deephaven-core that referenced this issue Mar 20, 2024
We've had a few recent issue/PRs dealing with python minimum version support: deephaven#5227, deephaven#5235, and deephaven#5271

Ultimately, we'd like a full matrix of testing support: deephaven#3724, deephaven#3725

In the more immediate term, there is a tool that I've verified is capable of catching these sorts of issues: https://github.com/netromdk/vermin. I've verified that it does catch deephaven#5227 and deephaven#5271, but does not catch deephaven#5235 (which seems like it is more of a runtime error).

This PR adds a quick CI job to verify a python minimum version of 3.8. The `# novermin` comment is necessary in a few locations where we've explicitly / manually worked around python minimum version support.
devinrsmith added a commit that referenced this issue Mar 21, 2024
We've had a few recent issue/PRs dealing with python minimum version support: #5227, #5235, and #5271

Ultimately, we'd like a full matrix of testing support: #3724, #3725

In the more immediate term, there is a tool that I've verified is capable of catching these sorts of issues: https://github.com/netromdk/vermin. I've verified that it does catch #5227 and #5271, but does not catch #5235 (which seems like it is more of a runtime error).

This PR adds a quick CI job to verify a python minimum version of 3.8. The `# novermin` comment is necessary in a few locations where we've explicitly / manually worked around python minimum version support.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

No branches or pull requests

1 participant