Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

tools: upgrade all python packages & add 'pip-requirements' action #12193

Merged
merged 3 commits into from
Sep 12, 2023
Merged

tools: upgrade all python packages & add 'pip-requirements' action #12193

merged 3 commits into from
Sep 12, 2023

Conversation

jaci-nordic
Copy link
Contributor

@jaci-nordic jaci-nordic commented Aug 31, 2023 •
edited by thst-nordic
Loading

pip-requirements GH Action:

  • The workflow merges the requirements from nrf, mcuboot and zephyr
  • Checks if the requirements-fixed and new generated file match
  • Packages that where only in the current requirements-fixed are added to requirements-ci.txt

python upgrades

  • ZCBOR commit was pulled in from modules/zcbor/tests: Fix command line parameter order #12016 to fix twister test.

  • Following packages will be added to requirements-fixed.txt with this change

    Deprecated
    Markdown
    PyGithub
    PyJWT
    PyNaCl
    azure-core
    azure-storage-blob
    capstone
    dill
    editdistance
    exceptiongroup
    gitlint-core
    graphviz
    grpcio
    importlib-resources
    isodate
    jsonschema-specifications
    lark
    libusb-package
    m2r2
    mistune
    msgpack
    pathspec
    pkgutil_resolve_name
    platformdirs
    pypng
    referencing
    rpds-py
    sphinx-copybutton
    sphinx-markdown-tables
    sphinx-ncs-theme
    sphinx-notfound-page
    sphinx-togglebutton
    sphinxcontrib-jquery
    sphinxcontrib-plantuml

@github-actions github-actions bot added the changelog-entry-required Update changelog before merge. Remove label if entry is not needed or already added. label Aug 31, 2023
@NordicBuilder
Copy link
Contributor

NordicBuilder commented Aug 31, 2023 •
edited
Loading

Test specification

CI/Jenkins/NRF

  • Integration Platforms

CI/Jenkins/integration

Test Module File based changes Manually selected West overwrite
desktop52_verification X
test-ci-nrfconnect-boot-fw-update X
test-fw-nrfconnect-apps X
test-fw-nrfconnect-ble X
test-fw-nrfconnect-ble_mesh X
test-fw-nrfconnect-ble_samples X
test-fw-nrfconnect-chip X
test-fw-nrfconnect-fem X
test-fw-nrfconnect-nfc X
test-fw-nrfconnect-nrf-iot_cloud X
test-fw-nrfconnect-nrf-iot_lwm2m X
test-fw-nrfconnect-nrf-iot_mosh X
test-fw-nrfconnect-nrf-iot_positioning X
test-fw-nrfconnect-nrf-iot_samples X
test-fw-nrfconnect-nrf-iot_serial_lte_modem X
test-fw-nrfconnect-nrf-iot_thingy91 X
test-fw-nrfconnect-nrf-iot_zephyr_lwm2m X
test-fw-nrfconnect-nrf_crypto X
test-fw-nrfconnect-proprietary_esb X
test-fw-nrfconnect-rpc X
test-fw-nrfconnect-rs X
test-fw-nrfconnect-tfm X
test-fw-nrfconnect-thread X
test-fw-nrfconnect-zigbee X
test-sdk-audio X
test-sdk-find-my X
test-sdk-homekit X
test-sdk-sidewalk X
test-sdk-wifi X

All integration tests: added because 'CI-all-test' was added manually

Detailed information of selected test modules

Note: This message is automatically posted and updated by the CI

@thst-nordic thst-nordic changed the title tools: adding pip requirements check workflow tools: upgrade all python packages & add 'pip-requirements' action Aug 31, 2023
thst-nordic
thst-nordic approved these changes Aug 31, 2023
View reviewed changes
Copy link
Contributor

@thst-nordic thst-nordic left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

bot editing of the requirements-fixed.txt file will come in a future PR.
We have it working, but not ready for this week.

@thst-nordic thst-nordic added this to the 2.5.0 milestone Aug 31, 2023
@thst-nordic thst-nordic added the CI-all-test Run All integration tests label Sep 1, 2023
tejlmand
tejlmand requested changes Sep 1, 2023
View reviewed changes
Copy link
Contributor

@tejlmand tejlmand left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Not approving a PR which may cause PRs to fail for reason that are completely unrelated to the changes in the PR itself.

If the requirements-fixed.txt PR goes in, then we can suddenly have 10 PRs that fails for the same reason, and which author of those 10 should fix the failure ?
Or even worse, all 10 authors waste time to investigate and fix the same error.

Please use a daily cron job, or similar for handling this issue.
Current proposed solution will create a lot of noise for everyone else.

scripts/requirements-build.txt Outdated Show resolved Hide resolved
scripts/requirements.txt Outdated
@@ -1,2 +1,4 @@
-r requirements-base.txt
-r requirements-build.txt
-r requirements-ci.txt
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

why must all developers install requirements for CI ?
https://developer.nordicsemi.com/nRF_Connect_SDK/doc/latest/nrf/installation/installing.html#install-additional-python-dependencies

Aren't the requirements-ci.txt intended for those setting up CI, and regular developers can ignore pip-packages referenced in this file ?

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@jaci-nordic removed it from requirements.txt and added to to the pip-compile command in compile-requirements.txt

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

could you please fixup this commit: 7851b12
to avoid adding the lines in the first place.

It's annoying when reviewing PRs or doing a blame later, to have lines added in one commit, which are then removed again in the next commit.

In this case, those lines:
https://github.com/nrfconnect/sdk-nrf/blob/7851b12da6f944c7fbeb9d749f66f788f25fc2b5/scripts/requirements.txt#L3-L4
are added in this commit: 7851b12
and removed in this commit: 0176f8f

.github/workflows/pip-requirements.yml Show resolved Hide resolved
@tejlmand
Copy link
Contributor

tejlmand commented Sep 4, 2023

If the requirements-fixed.txt PR goes in, then we can suddenly have 10 PRs that fails for the same reason, and which author of those 10 should fix the failure ?
Or even worse, all 10 authors waste time to investigate and fix the same error.

Seems I've overlooked this line:
https://github.com/nrfconnect/sdk-nrf/blob/e76e113d767bfc5a8daf52851bf79c55416be447/.github/workflows/pip-requirements.yml#L9-L10

so no concerns wrt. when this workflow runs.

Please reply to other comments, as they are still valid.

@jaci-nordic jaci-nordic force-pushed the pipRequirements branch 3 times, most recently from d4a1e3b to 8e663c6 Compare September 5, 2023 11:56
balaji-nordic pushed a commit to balaji-nordic/sdk-nrf that referenced this pull request Jun 5, 2024
deps: Install latest zcbor
e4e2fd2 
The code changes in ncs/main need newest zcbor. Hence
upgrading.
See nrfconnect#12193

Signed-off-by: Balaji Srinivasan <balaji.srinivasan@nordicsemi.no>
balaji-nordic pushed a commit to balaji-nordic/sdk-nrf that referenced this pull request Jun 6, 2024
deps: Install latest zcbor
70ce1db 
The code changes in ncs/main need newest zcbor. Hence
upgrading.
See nrfconnect#12193

Signed-off-by: Balaji Srinivasan <balaji.srinivasan@nordicsemi.no>
balaji-nordic pushed a commit to balaji-nordic/sdk-nrf that referenced this pull request Jun 7, 2024
deps: Install latest zcbor
efce6bc 
The code changes in ncs/main need newest zcbor. Hence
upgrading.
See nrfconnect#12193

Signed-off-by: Balaji Srinivasan <balaji.srinivasan@nordicsemi.no>
balaji-nordic pushed a commit to balaji-nordic/sdk-nrf that referenced this pull request Jun 8, 2024
deps: Install latest zcbor
68f013e 
The code changes in ncs/main need newest zcbor. Hence
upgrading.
See nrfconnect#12193

Signed-off-by: Balaji Srinivasan <balaji.srinivasan@nordicsemi.no>
balaji-nordic pushed a commit to balaji-nordic/sdk-nrf that referenced this pull request Jun 9, 2024
deps: Install latest zcbor
b4ff1b6 
The code changes in ncs/main need newest zcbor. Hence
upgrading.
See nrfconnect#12193

Signed-off-by: Balaji Srinivasan <balaji.srinivasan@nordicsemi.no>
balaji-nordic pushed a commit to balaji-nordic/sdk-nrf that referenced this pull request Jun 11, 2024
deps: Install latest zcbor
0624eb5 
The code changes in ncs/main need newest zcbor. Hence
upgrading.
See nrfconnect#12193

Signed-off-by: Balaji Srinivasan <balaji.srinivasan@nordicsemi.no>
balaji-nordic pushed a commit to balaji-nordic/sdk-nrf that referenced this pull request Jun 12, 2024
deps: Install latest zcbor
7d647f6 
The code changes in ncs/main need newest zcbor. Hence
upgrading.
See nrfconnect#12193

Signed-off-by: Balaji Srinivasan <balaji.srinivasan@nordicsemi.no>
balaji-nordic pushed a commit to balaji-nordic/sdk-nrf that referenced this pull request Jun 13, 2024
deps: Install latest zcbor
72d6b73 
The code changes in ncs/main need newest zcbor. Hence
upgrading.
See nrfconnect#12193

Signed-off-by: Balaji Srinivasan <balaji.srinivasan@nordicsemi.no>
balaji-nordic pushed a commit to balaji-nordic/sdk-nrf that referenced this pull request Jun 14, 2024
deps: Install latest zcbor
289ad39 
The code changes in ncs/main need newest zcbor. Hence
upgrading.
See nrfconnect#12193

Signed-off-by: Balaji Srinivasan <balaji.srinivasan@nordicsemi.no>
balaji-nordic pushed a commit to balaji-nordic/sdk-nrf that referenced this pull request Jun 15, 2024
deps: Install latest zcbor
e4d6b08 
The code changes in ncs/main need newest zcbor. Hence
upgrading.
See nrfconnect#12193

Signed-off-by: Balaji Srinivasan <balaji.srinivasan@nordicsemi.no>
balaji-nordic pushed a commit to balaji-nordic/sdk-nrf that referenced this pull request Jun 16, 2024
deps: Install latest zcbor
4617eb6 
The code changes in ncs/main need newest zcbor. Hence
upgrading.
See nrfconnect#12193

Signed-off-by: Balaji Srinivasan <balaji.srinivasan@nordicsemi.no>
balaji-nordic pushed a commit to balaji-nordic/sdk-nrf that referenced this pull request Jun 17, 2024
deps: Install latest zcbor
913b73b 
The code changes in ncs/main need newest zcbor. Hence
upgrading.
See nrfconnect#12193

Signed-off-by: Balaji Srinivasan <balaji.srinivasan@nordicsemi.no>
balaji-nordic pushed a commit to balaji-nordic/sdk-nrf that referenced this pull request Jun 18, 2024
deps: Install latest zcbor
0cbec6e 
The code changes in ncs/main need newest zcbor. Hence
upgrading.
See nrfconnect#12193

Signed-off-by: Balaji Srinivasan <balaji.srinivasan@nordicsemi.no>
balaji-nordic pushed a commit to balaji-nordic/sdk-nrf that referenced this pull request Jun 19, 2024
deps: Install latest zcbor
c3044c6 
The code changes in ncs/main need newest zcbor. Hence
upgrading.
See nrfconnect#12193

Signed-off-by: Balaji Srinivasan <balaji.srinivasan@nordicsemi.no>
balaji-nordic pushed a commit to balaji-nordic/sdk-nrf that referenced this pull request Jun 20, 2024
deps: Install latest zcbor
f350c89 
The code changes in ncs/main need newest zcbor. Hence
upgrading.
See nrfconnect#12193

Signed-off-by: Balaji Srinivasan <balaji.srinivasan@nordicsemi.no>
balaji-nordic pushed a commit to balaji-nordic/sdk-nrf that referenced this pull request Jun 21, 2024
deps: Install latest zcbor
9fc2201 
The code changes in ncs/main need newest zcbor. Hence
upgrading.
See nrfconnect#12193

Signed-off-by: Balaji Srinivasan <balaji.srinivasan@nordicsemi.no>
balaji-nordic pushed a commit to balaji-nordic/sdk-nrf that referenced this pull request Jun 22, 2024
deps: Install latest zcbor
f9883dc 
The code changes in ncs/main need newest zcbor. Hence
upgrading.
See nrfconnect#12193

Signed-off-by: Balaji Srinivasan <balaji.srinivasan@nordicsemi.no>
balaji-nordic pushed a commit to balaji-nordic/sdk-nrf that referenced this pull request Jun 25, 2024
deps: Install latest zcbor
994e305 
The code changes in ncs/main need newest zcbor. Hence
upgrading.
See nrfconnect#12193

Signed-off-by: Balaji Srinivasan <balaji.srinivasan@nordicsemi.no>
balaji-nordic pushed a commit to balaji-nordic/sdk-nrf that referenced this pull request Jun 26, 2024
deps: Install latest zcbor
d9cd257 
The code changes in ncs/main need newest zcbor. Hence
upgrading.
See nrfconnect#12193

Signed-off-by: Balaji Srinivasan <balaji.srinivasan@nordicsemi.no>
balaji-nordic pushed a commit to balaji-nordic/sdk-nrf that referenced this pull request Jun 27, 2024
deps: Install latest zcbor
adba8b5 
The code changes in ncs/main need newest zcbor. Hence
upgrading.
See nrfconnect#12193

Signed-off-by: Balaji Srinivasan <balaji.srinivasan@nordicsemi.no>
balaji-nordic pushed a commit to balaji-nordic/sdk-nrf that referenced this pull request Jun 28, 2024
deps: Install latest zcbor
25e876b 
The code changes in ncs/main need newest zcbor. Hence
upgrading.
See nrfconnect#12193

Signed-off-by: Balaji Srinivasan <balaji.srinivasan@nordicsemi.no>
balaji-nordic pushed a commit to balaji-nordic/sdk-nrf that referenced this pull request Jun 29, 2024
deps: Install latest zcbor
443b3f4 
The code changes in ncs/main need newest zcbor. Hence
upgrading.
See nrfconnect#12193

Signed-off-by: Balaji Srinivasan <balaji.srinivasan@nordicsemi.no>
balaji-nordic pushed a commit to balaji-nordic/sdk-nrf that referenced this pull request Jul 2, 2024
deps: Install latest zcbor
265c73d 
The code changes in ncs/main need newest zcbor. Hence
upgrading.
See nrfconnect#12193

Signed-off-by: Balaji Srinivasan <balaji.srinivasan@nordicsemi.no>
balaji-nordic pushed a commit to balaji-nordic/sdk-nrf that referenced this pull request Jul 3, 2024
deps: Install latest zcbor
9dfe22a 
The code changes in ncs/main need newest zcbor. Hence
upgrading.
See nrfconnect#12193

Signed-off-by: Balaji Srinivasan <balaji.srinivasan@nordicsemi.no>
balaji-nordic pushed a commit to balaji-nordic/sdk-nrf that referenced this pull request Jul 4, 2024
deps: Install latest zcbor
59b0e73 
The code changes in ncs/main need newest zcbor. Hence
upgrading.
See nrfconnect#12193

Signed-off-by: Balaji Srinivasan <balaji.srinivasan@nordicsemi.no>
balaji-nordic pushed a commit to balaji-nordic/sdk-nrf that referenced this pull request Jul 5, 2024
deps: Install latest zcbor
9d53117 
The code changes in ncs/main need newest zcbor. Hence
upgrading.
See nrfconnect#12193

Signed-off-by: Balaji Srinivasan <balaji.srinivasan@nordicsemi.no>
balaji-nordic pushed a commit to balaji-nordic/sdk-nrf that referenced this pull request Jul 6, 2024
deps: Install latest zcbor
b7ea07c 
The code changes in ncs/main need newest zcbor. Hence
upgrading.
See nrfconnect#12193

Signed-off-by: Balaji Srinivasan <balaji.srinivasan@nordicsemi.no>
balaji-nordic pushed a commit to balaji-nordic/sdk-nrf that referenced this pull request Jul 9, 2024
deps: Install latest zcbor
2e57200 
The code changes in ncs/main need newest zcbor. Hence
upgrading.
See nrfconnect#12193

Signed-off-by: Balaji Srinivasan <balaji.srinivasan@nordicsemi.no>
balaji-nordic pushed a commit to balaji-nordic/sdk-nrf that referenced this pull request Jul 10, 2024
deps: Install latest zcbor
021ebd0 
The code changes in ncs/main need newest zcbor. Hence
upgrading.
See nrfconnect#12193

Signed-off-by: Balaji Srinivasan <balaji.srinivasan@nordicsemi.no>
balaji-nordic pushed a commit to balaji-nordic/sdk-nrf that referenced this pull request Jul 11, 2024
deps: Install latest zcbor
241289a 
The code changes in ncs/main need newest zcbor. Hence
upgrading.
See nrfconnect#12193

Signed-off-by: Balaji Srinivasan <balaji.srinivasan@nordicsemi.no>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@thst-nordic thst-nordic thst-nordic approved these changes

@karhama karhama karhama approved these changes

@gmarull gmarull gmarull approved these changes

@tejlmand tejlmand tejlmand approved these changes

@oyvindronningstad oyvindronningstad oyvindronningstad approved these changes

@grho grho Awaiting requested review from grho

@shanthanordic shanthanordic Awaiting requested review from shanthanordic

@ihansse ihansse Awaiting requested review from ihansse

Assignees
No one assigned
Labels
changelog-entry-required Update changelog before merge. Remove label if entry is not needed or already added. CI-all-test Run All integration tests
Projects
None yet
Milestone
2.5.0
Development

Successfully merging this pull request may close these issues.
8 participants
@jaci-nordic @NordicBuilder @tejlmand @oyvindronningstad @thst-nordic @gmarull @karhama @de-nordic