SRE-2452 rpms: Script to build RPMs from scratch #15419

ryon-jensen · 2024-10-29T00:57:55Z

Script to build all RPMs, including dependencies.

Before requesting gatekeeper:

Two review approvals and any prior change requests have been resolved.
Testing is complete and all tests passed or there is a reason documented in the PR why it should be force landed and forced-landing tag is set.
Features: (or Test-tag*) commit pragma was used or there is a reason documented that there are no appropriate tags for this PR.
Commit messages follows the guidelines outlined here.
Any tests skipped by the ticket being addressed have been run and passed in the PR.

Gatekeeper:

github-actions · 2024-10-29T00:58:13Z

Errors are Unable to load ticket data
https://daosio.atlassian.net/browse/SRE-2452

daosbuild1 · 2024-10-29T01:12:31Z

Test stage Unit Test on EL 8.8 completed with status FAILURE. https://build.hpdd.intel.com/job/daos-stack/job/daos/job/PR-15419/5/display/redirect

daosbuild1 · 2024-10-29T01:12:45Z

Test stage NLT on EL 8.8 completed with status FAILURE. https://build.hpdd.intel.com/job/daos-stack/job/daos/job/PR-15419/5/display/redirect

daosbuild1 · 2024-10-29T01:22:15Z

Test stage NLT on EL 8.8 completed with status FAILURE. https://build.hpdd.intel.com/job/daos-stack/job/daos/job/PR-15419/9/display/redirect

daosbuild1 · 2024-10-29T01:45:56Z

Test stage Unit Test bdev on EL 8.8 completed with status FAILURE. https://build.hpdd.intel.com/job/daos-stack/job/daos/job/PR-15419/20/display/redirect

daosbuild1 · 2024-10-29T01:45:56Z

Test stage NLT on EL 8.8 completed with status FAILURE. https://build.hpdd.intel.com/job/daos-stack/job/daos/job/PR-15419/20/display/redirect

daosbuild1 · 2024-10-29T01:45:57Z

Test stage Unit Test with memcheck on EL 8.8 completed with status FAILURE. https://build.hpdd.intel.com/job/daos-stack/job/daos/job/PR-15419/20/display/redirect

daosbuild1 · 2024-10-29T01:45:58Z

Test stage Unit Test on EL 8.8 completed with status FAILURE. https://build.hpdd.intel.com/job/daos-stack/job/daos/job/PR-15419/20/display/redirect

daosbuild1 · 2024-10-29T01:45:59Z

Test stage Unit Test bdev with memcheck on EL 8.8 completed with status FAILURE. https://build.hpdd.intel.com/job/daos-stack/job/daos/job/PR-15419/20/display/redirect

daosbuild1 · 2024-10-29T02:16:27Z

Test stage Unit Test bdev on EL 8.8 completed with status FAILURE. https://build.hpdd.intel.com/job/daos-stack/job/daos/job/PR-15419/23/display/redirect

daosbuild1 · 2024-10-29T02:47:55Z

Test stage Unit Test bdev on EL 8.8 completed with status FAILURE. https://build.hpdd.intel.com/job/daos-stack/job/daos/job/PR-15419/24/display/redirect

daosbuild1 · 2024-10-30T01:36:21Z

Test stage NLT on EL 8.8 completed with status FAILURE. https://build.hpdd.intel.com/job/daos-stack/job/daos/job/PR-15419/31/display/redirect

daosbuild1 · 2024-10-30T01:36:23Z

Test stage Unit Test with memcheck on EL 8.8 completed with status FAILURE. https://build.hpdd.intel.com/job/daos-stack/job/daos/job/PR-15419/31/display/redirect

daosbuild1 · 2024-10-30T01:36:24Z

Test stage Unit Test on EL 8.8 completed with status FAILURE. https://build.hpdd.intel.com/job/daos-stack/job/daos/job/PR-15419/31/display/redirect

daosbuild1 · 2024-10-30T01:36:25Z

Test stage Unit Test bdev with memcheck on EL 8.8 completed with status FAILURE. https://build.hpdd.intel.com/job/daos-stack/job/daos/job/PR-15419/31/display/redirect

daosbuild1 · 2024-10-30T01:36:26Z

Test stage Unit Test bdev on EL 8.8 completed with status FAILURE. https://build.hpdd.intel.com/job/daos-stack/job/daos/job/PR-15419/31/display/redirect

.github/workflows/test_rpm_from_scratch_process.yml

Script to build all RPMs, including dependencies on RockyLinux 8 and 9. Introduces a new GHA workflow file for running the script. Signed-off-by: Ryon Jensen <ryon.jensen@intel.com> Doc-only: true

utils/build_rpms_from_scratch.sh

Signed-off-by: Ryon Jensen <ryon.jensen@intel.com> Doc-only: true

Signed-off-by: Ryon Jensen <ryon.jensen@intel.com>

daosbuild1 · 2024-11-02T20:28:29Z

Test stage Functional Hardware Medium Verbs Provider completed with status FAILURE. https://build.hpdd.intel.com/job/daos-stack/job/daos/job/PR-15419/43/display/redirect

daosbuild1 · 2024-11-02T20:29:53Z

Test stage Functional Hardware Medium completed with status FAILURE. https://build.hpdd.intel.com/job/daos-stack/job/daos/job/PR-15419/43/display/redirect

utils/build_rpms_from_scratch.sh

brianjmurrell · 2024-11-04T17:16:15Z

utils/build_rpms_from_scratch.sh

+      rpmdevtools \
+      createrepo_c \
+      make \
+      python3-Sphinx


Any idea what are requiring this? If it's failing rpmbuilds that's a packaging bug.

There's a comment for python3-Sphinx:

# python3-Sphinx is needed by dpdk. The file /usr/bin/sphinx-build is listed as a dependency in the spec file, # however, zypper isn't able to resolve a filename to a package and the scripting to handle that case is pretty # messy, especially because "zypper what-provides" actually returns three options for sphinx-build. So, forgoing # automation here and simply hardcoding the dependency.

I recognize it's not ideal, but weighing the effort and complexity (at least for me) for managing the dependency dynamically compared to hardcoding it and dealing with the impact on maintainability, I chose this approach. Definitely open to suggestions.

So that dpdk needs it means that the BuildRequires: in dpdk.spec should specify it. Did you get a failure to build dpdk without it? If you had a link to the failure that'd be awesome.

BuildRequires: specifies /usr/bin/sphinx-build, however, zypper doesn't resolve filenames to packages to install.

The other approach is that even if you want to continue to use zypper for the bulk of this script, recognize that zypper is inferior (or we are just ignorant) in regards to installing build-dependencies and install and use dnf for that. Perhaps if we are more obvious about this short-coming (or our ignorance) in zypper, somebody with more knowledge of zypper that is using and looking at this script can find/suggest a better way.

But I think by simply adding python3-Sphinx as an exception to the package install list here we are making the issue much (much!) more subtle than if we were to use dnf with a nice big comment about why we are using it.

utils/build_rpms_from_scratch.sh

brianjmurrell · 2024-11-04T17:32:05Z

utils/build_rpms_from_scratch.sh

+    git clone https://github.com/daos-stack/"$repo_name".git
+    cd "$repo_name"
+      install_dependencies "$repo_name".spec
+      make rpms RPM_BUILD_OPTIONS="--nocheck" # --nocheck = don't run tests during rpm build process


Suggested change

make rpms RPM_BUILD_OPTIONS="--nocheck" # --nocheck = don't run tests during rpm build process

MAKEARGS=()

if [ -f Makefile-rpm.mk ]; then

MAKEARGS=(-f Makefile-rpm.mk)

fi

if [ -d utils/rpms ]; then

MAKEARGS+=(-C utils/rpms)

fi

make "${MAKEARGS[@]}" rpms RPM_BUILD_OPTIONS="--nocheck" # --nocheck = don't run tests during rpm build process

will let you use this for raft and daos also and be future-proof against any other project that utilizes these exceptions.

I think it's a little more complicated than this. build_and_copy_rpm also clones the repo and because raft is a submodule of daos, the daos repo must be cloned first, then raft rpms built, then daos rpms can be built. So a bit of a chicken and egg issue with the way I wrote build_and_copy_rpms. I appreciate the help trying to remove the redundancy, though. I've probably already spent too much time trying to figure out how to remove the duplicate code without making things too complicated.

I think it's a little more complicated than this.

I don't think it is.

build_and_copy_rpm also clones the repo and because raft is a submodule of daos, the daos repo must be cloned first, then raft rpms built, then daos rpms can be built.

You can clone and build the raft repo independently of daos just like you do the other dozen repos. There is no need to clone daos and build raft out of the daos clone.

My concern is getting the correct raft commit that's actually used for daos. The 'fun' of it being a submodule is it doesn't have to be the tip of master. Right now it looks like the submodule daos uses is 1 commit behind the tip of master.

Doc-only: true Signed-off-by: Ryon Jensen <ryon.jensen@intel.com>

brianjmurrell · 2024-11-04T18:12:34Z

.github/workflows/test_rpm_from_scratch_process.yml

+        os: ["rockylinux:8.6",
+             "rockylinux:8.8",
+             "rockylinux:8.10",
+             "rockylinux:9.2",
+             "rockylinux:9.3",
+             "opensuse/leap:15.4",
+             "opensuse/leap:15.5",
+             "opensuse/leap:15.6"]


I wonder if using our self-hosted runner and therefore our own docker hub/cache, you will find more of these older versions. @JohnMalmberg Would we have these old point releases in our docker runners?

On the docker runners we are only keeping rockylinux8 and 9 up to date, no dot releases.
We have opensuse/leap:15.[2-5] and opensuse/leap-dnf:15.[4-5]

Just updated so that soon they will have leap and leap-dnf 15.6 and Fedora:41

On the docker runners we are only keeping rockylinux8 and 9 up to date, no dot releases.

This is a good case why it would be useful if we were keeping the EL point releases in our docker hub.

Doc-only: true Signed-off-by: Ryon Jensen <ryon.jensen@intel.com>

- DRY-ed some stuff - Fixed use of wrong variable - Fixed miss of directory change Doc-only: true Signed-off-by: Ryon Jensen <ryon.jensen@intel.com>

Doc-only: true Signed-off-by: Ryon Jensen <ryon.jensen@intel.com>

- matching of el major version for Power Tools repo was wrong - Added fail-fast: false (at least for now) Doc-only: true Signed-off-by: Ryon Jensen <ryon.jensen@intel.com>

- Not sure why I was using actions/checkout@v3 - bumped to v4 - Removed leap 15.6 for now. Having issues with it building - A little reorganization - Removed accidental add of file Doc-only: true Signed-off-by: Ryon Jensen <ryon.jensen@intel.com>

.github/workflows/test_rpm_from_scratch_process.yml

+
+    steps:
+      - name: Checkout code
+        uses: actions/checkout@v4


ryon-jensen force-pushed the ryon-jensen/rpm_building branch from f580fdf to f0a36da Compare October 30, 2024 11:41

github-advanced-security bot found potential problems Oct 30, 2024

View reviewed changes

.github/workflows/test_rpm_from_scratch_process.yml Fixed Show fixed Hide fixed

ryon-jensen marked this pull request as ready for review October 30, 2024 12:25

ryon-jensen requested review from brianjmurrell, grom72, janekmi, JohnMalmberg, jzarzycki and osalyk October 30, 2024 12:25

SRE-2452 rpms: Script to build RPMs from scratch

9df01c7

Script to build all RPMs, including dependencies on RockyLinux 8 and 9. Introduces a new GHA workflow file for running the script. Signed-off-by: Ryon Jensen <ryon.jensen@intel.com> Doc-only: true

ryon-jensen force-pushed the ryon-jensen/rpm_building branch from f0a36da to 9df01c7 Compare October 30, 2024 12:33

brianjmurrell requested changes Oct 30, 2024

View reviewed changes

utils/build_rpms_from_scratch.sh Outdated Show resolved Hide resolved

utils/build_rpms_from_scratch.sh Outdated Show resolved Hide resolved

utils/build_rpms_from_scratch.sh Outdated Show resolved Hide resolved

utils/build_rpms_from_scratch.sh Outdated Show resolved Hide resolved

Review feedback items

c717e40

Signed-off-by: Ryon Jensen <ryon.jensen@intel.com> Doc-only: true

ryon-jensen force-pushed the ryon-jensen/rpm_building branch from 0405f72 to c717e40 Compare October 30, 2024 19:48

ryon-jensen force-pushed the ryon-jensen/rpm_building branch from 5c6335f to 7aaed8c Compare October 30, 2024 22:47

potential fix

93b3ad5

Signed-off-by: Ryon Jensen <ryon.jensen@intel.com> Doc-only: true

ryon-jensen force-pushed the ryon-jensen/rpm_building branch from 7aaed8c to 93b3ad5 Compare October 30, 2024 22:49

ryon-jensen requested a review from brianjmurrell October 31, 2024 00:19

Added support for opensuse/leap

74df489

Signed-off-by: Ryon Jensen <ryon.jensen@intel.com>

brianjmurrell requested changes Nov 4, 2024

View reviewed changes

Attempting not failing if docker image doesn't exist

fe221af

Doc-only: true Signed-off-by: Ryon Jensen <ryon.jensen@intel.com>

ryon-jensen force-pushed the ryon-jensen/rpm_building branch from 3fd1c84 to fe221af Compare November 4, 2024 18:02

brianjmurrell reviewed Nov 4, 2024

View reviewed changes

ryon-jensen added 5 commits November 4, 2024 11:41

Other feedback and fixes

d246fa2

Doc-only: true Signed-off-by: Ryon Jensen <ryon.jensen@intel.com>

Other feedback and fixes

3b66354

- DRY-ed some stuff - Fixed use of wrong variable - Fixed miss of directory change Doc-only: true Signed-off-by: Ryon Jensen <ryon.jensen@intel.com>

Added more OS checking in script

3a65e2f

Doc-only: true Signed-off-by: Ryon Jensen <ryon.jensen@intel.com>

Fixes

aba44cc

- matching of el major version for Power Tools repo was wrong - Added fail-fast: false (at least for now) Doc-only: true Signed-off-by: Ryon Jensen <ryon.jensen@intel.com>

Fixes

af4c2cd

- Not sure why I was using actions/checkout@v3 - bumped to v4 - Removed leap 15.6 for now. Having issues with it building - A little reorganization - Removed accidental add of file Doc-only: true Signed-off-by: Ryon Jensen <ryon.jensen@intel.com>

github-advanced-security bot found potential problems Nov 9, 2024

View reviewed changes

.github/workflows/test_rpm_from_scratch_process.yml

steps:

- name: Checkout code

uses: actions/checkout@v4

Check warning

Code scanning / Scorecard

Pinned-Dependencies Medium test

score is 1: GitHub-owned GitHubAction not pinned by hash
Click Remediation section below to solve this issue

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

SRE-2452 rpms: Script to build RPMs from scratch #15419

SRE-2452 rpms: Script to build RPMs from scratch #15419

ryon-jensen commented Oct 29, 2024

github-actions bot commented Oct 29, 2024

daosbuild1 commented Oct 29, 2024

daosbuild1 commented Oct 29, 2024

daosbuild1 commented Oct 29, 2024

daosbuild1 commented Oct 29, 2024

daosbuild1 commented Oct 29, 2024

daosbuild1 commented Oct 29, 2024

daosbuild1 commented Oct 29, 2024

daosbuild1 commented Oct 29, 2024

daosbuild1 commented Oct 29, 2024

daosbuild1 commented Oct 29, 2024

daosbuild1 commented Oct 30, 2024

daosbuild1 commented Oct 30, 2024

daosbuild1 commented Oct 30, 2024

daosbuild1 commented Oct 30, 2024

daosbuild1 commented Oct 30, 2024

daosbuild1 commented Nov 2, 2024

daosbuild1 commented Nov 2, 2024

brianjmurrell Nov 4, 2024

ryon-jensen Nov 4, 2024 •

edited

Loading

brianjmurrell Nov 4, 2024

ryon-jensen Nov 8, 2024

brianjmurrell Nov 12, 2024

brianjmurrell Nov 4, 2024

ryon-jensen Nov 8, 2024

brianjmurrell Nov 12, 2024

ryon-jensen Nov 14, 2024

brianjmurrell Nov 4, 2024

JohnMalmberg Nov 4, 2024

brianjmurrell Nov 4, 2024

-      make rpms RPM_BUILD_OPTIONS="--nocheck" # --nocheck = don't run tests during rpm build process
+      MAKEARGS=()
+      if [ -f Makefile-rpm.mk ]; then
+        MAKEARGS=(-f Makefile-rpm.mk)
+      fi
+      if [ -d utils/rpms ]; then
+        MAKEARGS+=(-C utils/rpms)
+      fi
+      make "${MAKEARGS[@]}" rpms RPM_BUILD_OPTIONS="--nocheck" # --nocheck = don't run tests during rpm build process

SRE-2452 rpms: Script to build RPMs from scratch #15419

Are you sure you want to change the base?

SRE-2452 rpms: Script to build RPMs from scratch #15419

Conversation

ryon-jensen commented Oct 29, 2024

Before requesting gatekeeper:

Gatekeeper:

github-actions bot commented Oct 29, 2024

daosbuild1 commented Oct 29, 2024

daosbuild1 commented Oct 29, 2024

daosbuild1 commented Oct 29, 2024

daosbuild1 commented Oct 29, 2024

daosbuild1 commented Oct 29, 2024

daosbuild1 commented Oct 29, 2024

daosbuild1 commented Oct 29, 2024

daosbuild1 commented Oct 29, 2024

daosbuild1 commented Oct 29, 2024

daosbuild1 commented Oct 29, 2024

daosbuild1 commented Oct 30, 2024

daosbuild1 commented Oct 30, 2024

daosbuild1 commented Oct 30, 2024

daosbuild1 commented Oct 30, 2024

daosbuild1 commented Oct 30, 2024

daosbuild1 commented Nov 2, 2024

daosbuild1 commented Nov 2, 2024

Choose a reason for hiding this comment

ryon-jensen Nov 4, 2024 • edited Loading

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

ryon-jensen Nov 4, 2024 •

edited

Loading