Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Add Client Version 2.7.0 #93

Merged
merged 1 commit into from
Aug 21, 2023
Merged

Add Client Version 2.7.0 #93

merged 1 commit into from
Aug 21, 2023

Conversation

ephraimbuddy
Copy link
Contributor

No description provided.

@potiuk
Copy link
Member

potiuk commented Aug 21, 2023

SMth wrong: import client where previously import airflow_client ? I guess some command line flag in the generator changed and it stopped generating the same "airflow_client" package.

@ephraimbuddy
Copy link
Contributor Author

SMth wrong: import client where previously import airflow_client ? I guess some command line flag in the generator changed and it stopped generating the same "airflow_client" package.

Yes. The sed command failed but I didn't look well:
+ find /Users/ephraimbuddy/projects/airflow-client-python/airflow_client/test -type f -name '*.py' -exec sed -i '' -e s/client/airflow_client.client/g '{}' + sed: can't read : No such file or directory

@ephraimbuddy
Copy link
Contributor Author

SMth wrong: import client where previously import airflow_client ? I guess some command line flag in the generator changed and it stopped generating the same "airflow_client" package.

Yes. The sed command failed but I didn't look well: + find /Users/ephraimbuddy/projects/airflow-client-python/airflow_client/test -type f -name '*.py' -exec sed -i '' -e s/client/airflow_client.client/g '{}' + sed: can't read : No such file or directory

To fix this, I had to uninstall gnu-sed which I installed sometime in the past

@potiuk
Copy link
Member

potiuk commented Aug 21, 2023

SMth wrong: import client where previously import airflow_client ? I guess some command line flag in the generator changed and it stopped generating the same "airflow_client" package.

Yes. The sed command failed but I didn't look well: + find /Users/ephraimbuddy/projects/airflow-client-python/airflow_client/test -type f -name '*.py' -exec sed -i '' -e s/client/airflow_client.client/g '{}' + sed: can't read : No such file or directory

To fix this, I had to uninstall gnu-sed which I installed sometime in the past

calls for an improvement in the chain :) . It's a bit strange we are using sed to correct those imports 🤯

@ephraimbuddy ephraimbuddy merged commit fad0465 into main Aug 21, 2023
5 checks passed
@ephraimbuddy ephraimbuddy deleted the release270 branch August 21, 2023 12:16
potiuk added a commit that referenced this pull request Jan 13, 2024
@potiuk
Housekeeping of Python client repository
b2030e0 
After working on reproducible package proparation for Python client
it's been found that the Python client repository had some extra
code added and docs and tests were placed in a wrong directory.

As of #93, we
have duplicated airflow_client code:

* airflow_client.client - this is the one that should be used
* airflow_client.airflow_client - this is the one that is added
  accidentally (and it is a copy of the `client` one)

Also `docs` and `tests` are generated in "airflow_client" package,
but this is wrong, both docs and tests should be at the top-level
of the repository.

This PR:

* deletes the extra code
* moves "docs" and "test" folders to the top-level of the project.
@potiuk potiuk mentioned this pull request Jan 13, 2024
Merged
potiuk added a commit that referenced this pull request Jan 13, 2024
@potiuk
Housekeeping of Python client repository
8cc9575 
After working on reproducible package proparation for Python client
it's been found that the Python client repository had some extra
code added and docs and tests were placed in a wrong directory.

As of #93, we
have duplicated airflow_client code:

* airflow_client.client - this is the one that should be used
* airflow_client.airflow_client - this is the one that is added
  accidentally (and it is a copy of the `client` one)

Also `docs` and `tests` are generated in "airflow_client" package,
but this is wrong, both docs and tests should be at the top-level
of the repository.

This PR:

* deletes the extra code
* moves "docs" and "test" folders to the top-level of the project.
potiuk added a commit that referenced this pull request Jan 13, 2024
@potiuk
Housekeeping of Python client repository
8eaf57b 
After working on reproducible package proparation for Python client
it's been found that the Python client repository had some extra
code added and docs and tests were placed in a wrong directory.

As of #93, we
have duplicated airflow_client code:

* airflow_client.client - this is the one that should be used
* airflow_client.airflow_client - this is the one that is added
  accidentally (and it is a copy of the `client` one)

Also `docs` and `tests` are generated in "airflow_client" package,
but this is wrong, both docs and tests should be at the top-level
of the repository.

This PR:

* deletes the extra code
* moves "docs" and "test" folders to the top-level of the project.
potiuk added a commit that referenced this pull request Jan 13, 2024
@potiuk
Housekeeping of Python client repository (#103)
bc7f81e 
After working on reproducible package proparation for Python client
it's been found that the Python client repository had some extra
code added and docs and tests were placed in a wrong directory.

As of #93, we
have duplicated airflow_client code:

* airflow_client.client - this is the one that should be used
* airflow_client.airflow_client - this is the one that is added
  accidentally (and it is a copy of the `client` one)

Also `docs` and `tests` are generated in "airflow_client" package,
but this is wrong, both docs and tests should be at the top-level
of the repository.

This PR:

* deletes the extra code
* moves "docs" and "test" folders to the top-level of the project.
@potiuk potiuk mentioned this pull request Jan 13, 2024
Merged
potiuk added a commit to potiuk/airflow that referenced this pull request Jan 14, 2024
@potiuk
Generate Python client in reproducible way
57b5252 
Client source code and package generation was done using the code
generated and committed to `airflow-client-python` and while the
repository with such code is useful to have, it's just a convenience
repo, because all sources are (and should be) generated from the
API specification which is present in the Airflow repository.

This also made the reproducible builds and package generation not really
possible, because we never knew if the source generated in the
`airflow-client-python` repository has been generated and not tampered
with.

While implementing it, it turned out that there were some issues in
the past that nade our client generation somewhat broken..

* In 2.7.0 python client, we added the same code twice
  (See apache/airflow-client-python#93) on
  top of "airflow_client.client" package, we also added copy of the
  API client generated in "airflow_client.airflow_client" - that was
  likely due to bad bash scripts and tools that were used to generate
  it and errors during generation the clients.

* We used to generate the code for "client" package and then moved
  the "client" package to "airflow_client.client" package, while
  manually modifying imports with `sed` (!?). That was likely due to
  limitations in some old version of the client generator. However the
  client generator we use now is capable of generating code directly in
  the "airflow_client.client" package.

* We also manually (via pre-commit) added Apache Licence to the
  generated files. Whieh was completely unnecessary, because ASF rules
  do not require licence headers to be added to code automatically
  generated from a code that already has ASF licence.

* We also generated source tarball packages from such generated code,
  which was completely unnecessary - because sdist packages are already
  fulfilling all the reqirements of such source pacakges - the code
  in the packages is enough to build the package from the sources and
  it does not contain any binary code, moreover the code is generated
  out of the API specificiation, which means that anyone can take
  the code and genearate the pacakged software from just sources in
  sdist. Similarly as in case of provider packages, we do not need
  to produce separate -source.tar.gz files.

This PR fixes all of it.

First of all the source that lands in the source repository
`airflow-client-python` and sdist/wheel packages are generated directly
from the openapi specification.

They are generated using breeze release_management command from airflow
source  tagged with specific tag in the Airflow repo (including the
source of reproducible build date that is updated together with airflow
release notes. This means that any PMC member can regenerate packages
(binary identical) straight from the Airflow repository - without
going through "airflow-client-python" repository.

No source tarball is generated - it is not needed, sdist is enough.

The `test_python_client.py` has been also moved over to Airflow repo
and updated with handling case when expose_config is not enabled and
it is used to automatically test the API client after it has been
generated.
potiuk added a commit to apache/airflow that referenced this pull request Jan 14, 2024
@potiuk
Generate Python client in reproducible way (#36763)
9787440 
Client source code and package generation was done using the code
generated and committed to `airflow-client-python` and while the
repository with such code is useful to have, it's just a convenience
repo, because all sources are (and should be) generated from the
API specification which is present in the Airflow repository.

This also made the reproducible builds and package generation not really
possible, because we never knew if the source generated in the
`airflow-client-python` repository has been generated and not tampered
with.

While implementing it, it turned out that there were some issues in
the past that nade our client generation somewhat broken..

* In 2.7.0 python client, we added the same code twice
  (See apache/airflow-client-python#93) on
  top of "airflow_client.client" package, we also added copy of the
  API client generated in "airflow_client.airflow_client" - that was
  likely due to bad bash scripts and tools that were used to generate
  it and errors during generation the clients.

* We used to generate the code for "client" package and then moved
  the "client" package to "airflow_client.client" package, while
  manually modifying imports with `sed` (!?). That was likely due to
  limitations in some old version of the client generator. However the
  client generator we use now is capable of generating code directly in
  the "airflow_client.client" package.

* We also manually (via pre-commit) added Apache Licence to the
  generated files. Whieh was completely unnecessary, because ASF rules
  do not require licence headers to be added to code automatically
  generated from a code that already has ASF licence.

* We also generated source tarball packages from such generated code,
  which was completely unnecessary - because sdist packages are already
  fulfilling all the reqirements of such source pacakges - the code
  in the packages is enough to build the package from the sources and
  it does not contain any binary code, moreover the code is generated
  out of the API specificiation, which means that anyone can take
  the code and genearate the pacakged software from just sources in
  sdist. Similarly as in case of provider packages, we do not need
  to produce separate -source.tar.gz files.

This PR fixes all of it.

First of all the source that lands in the source repository
`airflow-client-python` and sdist/wheel packages are generated directly
from the openapi specification.

They are generated using breeze release_management command from airflow
source  tagged with specific tag in the Airflow repo (including the
source of reproducible build date that is updated together with airflow
release notes. This means that any PMC member can regenerate packages
(binary identical) straight from the Airflow repository - without
going through "airflow-client-python" repository.

No source tarball is generated - it is not needed, sdist is enough.

The `test_python_client.py` has been also moved over to Airflow repo
and updated with handling case when expose_config is not enabled and
it is used to automatically test the API client after it has been
generated.
ephraimbuddy pushed a commit to apache/airflow that referenced this pull request Jan 15, 2024
@potiuk @ephraimbuddy
Generate Python client in reproducible way (#36763)
42263f0 
Client source code and package generation was done using the code
generated and committed to `airflow-client-python` and while the
repository with such code is useful to have, it's just a convenience
repo, because all sources are (and should be) generated from the
API specification which is present in the Airflow repository.

This also made the reproducible builds and package generation not really
possible, because we never knew if the source generated in the
`airflow-client-python` repository has been generated and not tampered
with.

While implementing it, it turned out that there were some issues in
the past that nade our client generation somewhat broken..

* In 2.7.0 python client, we added the same code twice
  (See apache/airflow-client-python#93) on
  top of "airflow_client.client" package, we also added copy of the
  API client generated in "airflow_client.airflow_client" - that was
  likely due to bad bash scripts and tools that were used to generate
  it and errors during generation the clients.

* We used to generate the code for "client" package and then moved
  the "client" package to "airflow_client.client" package, while
  manually modifying imports with `sed` (!?). That was likely due to
  limitations in some old version of the client generator. However the
  client generator we use now is capable of generating code directly in
  the "airflow_client.client" package.

* We also manually (via pre-commit) added Apache Licence to the
  generated files. Whieh was completely unnecessary, because ASF rules
  do not require licence headers to be added to code automatically
  generated from a code that already has ASF licence.

* We also generated source tarball packages from such generated code,
  which was completely unnecessary - because sdist packages are already
  fulfilling all the reqirements of such source pacakges - the code
  in the packages is enough to build the package from the sources and
  it does not contain any binary code, moreover the code is generated
  out of the API specificiation, which means that anyone can take
  the code and genearate the pacakged software from just sources in
  sdist. Similarly as in case of provider packages, we do not need
  to produce separate -source.tar.gz files.

This PR fixes all of it.

First of all the source that lands in the source repository
`airflow-client-python` and sdist/wheel packages are generated directly
from the openapi specification.

They are generated using breeze release_management command from airflow
source  tagged with specific tag in the Airflow repo (including the
source of reproducible build date that is updated together with airflow
release notes. This means that any PMC member can regenerate packages
(binary identical) straight from the Airflow repository - without
going through "airflow-client-python" repository.

No source tarball is generated - it is not needed, sdist is enough.

The `test_python_client.py` has been also moved over to Airflow repo
and updated with handling case when expose_config is not enabled and
it is used to automatically test the API client after it has been
generated.

(cherry picked from commit 9787440)
kosteev pushed a commit to GoogleCloudPlatform/composer-airflow that referenced this pull request Jul 18, 2024
@potiuk
Generate Python client in reproducible way (#36763)
6b2609c 
Client source code and package generation was done using the code
generated and committed to `airflow-client-python` and while the
repository with such code is useful to have, it's just a convenience
repo, because all sources are (and should be) generated from the
API specification which is present in the Airflow repository.

This also made the reproducible builds and package generation not really
possible, because we never knew if the source generated in the
`airflow-client-python` repository has been generated and not tampered
with.

While implementing it, it turned out that there were some issues in
the past that nade our client generation somewhat broken..

* In 2.7.0 python client, we added the same code twice
  (See apache/airflow-client-python#93) on
  top of "airflow_client.client" package, we also added copy of the
  API client generated in "airflow_client.airflow_client" - that was
  likely due to bad bash scripts and tools that were used to generate
  it and errors during generation the clients.

* We used to generate the code for "client" package and then moved
  the "client" package to "airflow_client.client" package, while
  manually modifying imports with `sed` (!?). That was likely due to
  limitations in some old version of the client generator. However the
  client generator we use now is capable of generating code directly in
  the "airflow_client.client" package.

* We also manually (via pre-commit) added Apache Licence to the
  generated files. Whieh was completely unnecessary, because ASF rules
  do not require licence headers to be added to code automatically
  generated from a code that already has ASF licence.

* We also generated source tarball packages from such generated code,
  which was completely unnecessary - because sdist packages are already
  fulfilling all the reqirements of such source pacakges - the code
  in the packages is enough to build the package from the sources and
  it does not contain any binary code, moreover the code is generated
  out of the API specificiation, which means that anyone can take
  the code and genearate the pacakged software from just sources in
  sdist. Similarly as in case of provider packages, we do not need
  to produce separate -source.tar.gz files.

This PR fixes all of it.

First of all the source that lands in the source repository
`airflow-client-python` and sdist/wheel packages are generated directly
from the openapi specification.

They are generated using breeze release_management command from airflow
source  tagged with specific tag in the Airflow repo (including the
source of reproducible build date that is updated together with airflow
release notes. This means that any PMC member can regenerate packages
(binary identical) straight from the Airflow repository - without
going through "airflow-client-python" repository.

No source tarball is generated - it is not needed, sdist is enough.

The `test_python_client.py` has been also moved over to Airflow repo
and updated with handling case when expose_config is not enabled and
it is used to automatically test the API client after it has been
generated.

GitOrigin-RevId: 9787440593196881b481466aa6d3cca4408f99e5
kosteev pushed a commit to GoogleCloudPlatform/composer-airflow that referenced this pull request Sep 20, 2024
@potiuk
Generate Python client in reproducible way (#36763)
257b960 
Client source code and package generation was done using the code
generated and committed to `airflow-client-python` and while the
repository with such code is useful to have, it's just a convenience
repo, because all sources are (and should be) generated from the
API specification which is present in the Airflow repository.

This also made the reproducible builds and package generation not really
possible, because we never knew if the source generated in the
`airflow-client-python` repository has been generated and not tampered
with.

While implementing it, it turned out that there were some issues in
the past that nade our client generation somewhat broken..

* In 2.7.0 python client, we added the same code twice
  (See apache/airflow-client-python#93) on
  top of "airflow_client.client" package, we also added copy of the
  API client generated in "airflow_client.airflow_client" - that was
  likely due to bad bash scripts and tools that were used to generate
  it and errors during generation the clients.

* We used to generate the code for "client" package and then moved
  the "client" package to "airflow_client.client" package, while
  manually modifying imports with `sed` (!?). That was likely due to
  limitations in some old version of the client generator. However the
  client generator we use now is capable of generating code directly in
  the "airflow_client.client" package.

* We also manually (via pre-commit) added Apache Licence to the
  generated files. Whieh was completely unnecessary, because ASF rules
  do not require licence headers to be added to code automatically
  generated from a code that already has ASF licence.

* We also generated source tarball packages from such generated code,
  which was completely unnecessary - because sdist packages are already
  fulfilling all the reqirements of such source pacakges - the code
  in the packages is enough to build the package from the sources and
  it does not contain any binary code, moreover the code is generated
  out of the API specificiation, which means that anyone can take
  the code and genearate the pacakged software from just sources in
  sdist. Similarly as in case of provider packages, we do not need
  to produce separate -source.tar.gz files.

This PR fixes all of it.

First of all the source that lands in the source repository
`airflow-client-python` and sdist/wheel packages are generated directly
from the openapi specification.

They are generated using breeze release_management command from airflow
source  tagged with specific tag in the Airflow repo (including the
source of reproducible build date that is updated together with airflow
release notes. This means that any PMC member can regenerate packages
(binary identical) straight from the Airflow repository - without
going through "airflow-client-python" repository.

No source tarball is generated - it is not needed, sdist is enough.

The `test_python_client.py` has been also moved over to Airflow repo
and updated with handling case when expose_config is not enabled and
it is used to automatically test the API client after it has been
generated.

GitOrigin-RevId: 9787440593196881b481466aa6d3cca4408f99e5
kosteev pushed a commit to GoogleCloudPlatform/composer-airflow that referenced this pull request Nov 8, 2024
@potiuk
Generate Python client in reproducible way (#36763)
ffda72c 
Client source code and package generation was done using the code
generated and committed to `airflow-client-python` and while the
repository with such code is useful to have, it's just a convenience
repo, because all sources are (and should be) generated from the
API specification which is present in the Airflow repository.

This also made the reproducible builds and package generation not really
possible, because we never knew if the source generated in the
`airflow-client-python` repository has been generated and not tampered
with.

While implementing it, it turned out that there were some issues in
the past that nade our client generation somewhat broken..

* In 2.7.0 python client, we added the same code twice
  (See apache/airflow-client-python#93) on
  top of "airflow_client.client" package, we also added copy of the
  API client generated in "airflow_client.airflow_client" - that was
  likely due to bad bash scripts and tools that were used to generate
  it and errors during generation the clients.

* We used to generate the code for "client" package and then moved
  the "client" package to "airflow_client.client" package, while
  manually modifying imports with `sed` (!?). That was likely due to
  limitations in some old version of the client generator. However the
  client generator we use now is capable of generating code directly in
  the "airflow_client.client" package.

* We also manually (via pre-commit) added Apache Licence to the
  generated files. Whieh was completely unnecessary, because ASF rules
  do not require licence headers to be added to code automatically
  generated from a code that already has ASF licence.

* We also generated source tarball packages from such generated code,
  which was completely unnecessary - because sdist packages are already
  fulfilling all the reqirements of such source pacakges - the code
  in the packages is enough to build the package from the sources and
  it does not contain any binary code, moreover the code is generated
  out of the API specificiation, which means that anyone can take
  the code and genearate the pacakged software from just sources in
  sdist. Similarly as in case of provider packages, we do not need
  to produce separate -source.tar.gz files.

This PR fixes all of it.

First of all the source that lands in the source repository
`airflow-client-python` and sdist/wheel packages are generated directly
from the openapi specification.

They are generated using breeze release_management command from airflow
source  tagged with specific tag in the Airflow repo (including the
source of reproducible build date that is updated together with airflow
release notes. This means that any PMC member can regenerate packages
(binary identical) straight from the Airflow repository - without
going through "airflow-client-python" repository.

No source tarball is generated - it is not needed, sdist is enough.

The `test_python_client.py` has been also moved over to Airflow repo
and updated with handling case when expose_config is not enabled and
it is used to automatically test the API client after it has been
generated.

GitOrigin-RevId: 9787440593196881b481466aa6d3cca4408f99e5
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@potiuk potiuk potiuk approved these changes

@pierrejeambrun pierrejeambrun Awaiting requested review from pierrejeambrun

@jedcunningham jedcunningham Awaiting requested review from jedcunningham

Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.
2 participants
@ephraimbuddy @potiuk