DAOS-13224 build test: Build and test on EL9 #12176

brianjmurrell · 2023-05-20T16:04:22Z

Remove production of the version stash. It's not necessary any longer.

Add a Fixes commit pragma to allow a PR to fix more than one issue that
might be in the skip list.

Remove libfabric along with any daos packages prior to installing daos,
to ensure that the correct libfabric is installed.

Remove use of lsb_release as it's gone in EL9. Use /etc/os-release
instead.

s-nail replaces mail[x] for sending e-mail on EL9.

Remove ci/functional/required_packages.sh and move all functional test
dependency specifications into RPMs as Requires:.

Add a client-tests-mpich subpackage for mpich test dependencies.

Remove the "Scan EL 8 RPMs" stage due to Jenkinsfile size limitations.

Fixes: SRE-1899 DAOS-13225

Before requesting gatekeeper:

Two review approvals and any prior change requests have been resolved.
Testing is complete and all tests passed or there is a reason documented in the PR why it should be force landed and forced-landing tag is set.
- No code related to failed tests was modified
Features: (or Test-tag*) commit pragma was used or there is a reason documented that there are no appropriate tags for this PR.
Commit messages follows the guidelines outlined here.
Any tests skipped by the ticket being addressed have been run and passed in the PR.

Gatekeeper:

github-actions · 2023-05-20T16:04:48Z

Bug-tracker data:
Ticket title is 'Update DAOS RPM build for EL9'
Status is 'In Progress'
Errors are component not formatted correctly
https://daosio.atlassian.net/browse/DAOS-13224

daosbuild1

Style warning(s) for job https://build.hpdd.intel.com/job/daos-stack/job/daos/job/PR-12176/3/
Please review https://wiki.hpdd.intel.com/display/DC/Coding+Rules

FYI: Errors found in lines not modified in the patch:

utils/rpms/packaging/rpm_chrootbuild:107:
(lint) Use "${array[@]}" (with quotes) to prevent whitespace problems. [SC2048]
(lint) Use "${array[@]}" (with quotes) to prevent whitespace problems. [SC2048]

utils/rpms/packaging/rpm_chrootbuild:108:
(lint) eval negates the benefit of arrays. Drop eval to preserve whitespace/symbols (or eval as string). [SC2294]

ci/provisioning/post_provision_config_common_functions.sh

daosbuild1 · 2023-06-05T19:19:59Z

Test stage Python Bandit check completed with status FAILURE. https://build.hpdd.intel.com//job/daos-stack/job/daos/view/change-requests/job/PR-12176/84/execution/node/124/log

Updated patch

daosbuild1

Style warning(s) for job https://build.hpdd.intel.com/job/daos-stack/job/daos/job/PR-12176/84/
Please review https://wiki.hpdd.intel.com/display/DC/Coding+Rules

FYI: Errors found in lines not modified in the patch:

utils/rpms/packaging/rpm_chrootbuild:118:
(lint) Use "${array[@]}" (with quotes) to prevent whitespace problems. [SC2048]
(lint) Use "${array[@]}" (with quotes) to prevent whitespace problems. [SC2048]

utils/rpms/packaging/rpm_chrootbuild:119:
(lint) eval negates the benefit of arrays. Drop eval to preserve whitespace/symbols (or eval as string). [SC2294]

ci/provisioning/post_provision_config_common_functions.sh

utils/rpms/packaging/Dockerfile.mockbuild

daosbuild1 · 2023-06-05T19:20:23Z

Test stage checkpatch completed with status FAILURE. https://build.hpdd.intel.com//job/daos-stack/job/daos/view/change-requests/job/PR-12176/84/execution/node/171/log

daosbuild1 · 2023-06-06T01:38:39Z

Test stage Python Bandit check completed with status FAILURE. https://build.hpdd.intel.com//job/daos-stack/job/daos/view/change-requests/job/PR-12176/85/execution/node/133/log

Updated patch

daosbuild1

Style warning(s) for job https://build.hpdd.intel.com/job/daos-stack/job/daos/job/PR-12176/85/
Please review https://wiki.hpdd.intel.com/display/DC/Coding+Rules

FYI: Errors found in lines not modified in the patch:

utils/rpms/packaging/rpm_chrootbuild:118:
(lint) Use "${array[@]}" (with quotes) to prevent whitespace problems. [SC2048]
(lint) Use "${array[@]}" (with quotes) to prevent whitespace problems. [SC2048]

utils/rpms/packaging/rpm_chrootbuild:119:
(lint) eval negates the benefit of arrays. Drop eval to preserve whitespace/symbols (or eval as string). [SC2294]

ci/provisioning/post_provision_config_common_functions.sh

utils/rpms/packaging/Dockerfile.mockbuild

daosbuild1 · 2023-06-06T01:40:42Z

Test stage checkpatch completed with status FAILURE. https://build.hpdd.intel.com//job/daos-stack/job/daos/view/change-requests/job/PR-12176/85/execution/node/181/log

Updated patch

daosbuild1

Style warning(s) for job https://build.hpdd.intel.com/job/daos-stack/job/daos/job/PR-12176/86/
Please review https://wiki.hpdd.intel.com/display/DC/Coding+Rules

FYI: Errors found in lines not modified in the patch:

utils/rpms/packaging/rpm_chrootbuild:118:
(lint) Use "${array[@]}" (with quotes) to prevent whitespace problems. [SC2048]
(lint) Use "${array[@]}" (with quotes) to prevent whitespace problems. [SC2048]

utils/rpms/packaging/rpm_chrootbuild:119:
(lint) eval negates the benefit of arrays. Drop eval to preserve whitespace/symbols (or eval as string). [SC2294]

ci/provisioning/post_provision_config_common_functions.sh

utils/rpms/packaging/Dockerfile.mockbuild

daosbuild1 · 2023-06-06T01:51:18Z

Test stage checkpatch completed with status FAILURE. https://build.hpdd.intel.com//job/daos-stack/job/daos/view/change-requests/job/PR-12176/86/execution/node/146/log

daosbuild1 · 2023-06-06T06:22:13Z

Test stage Functional on EL 9 completed with status FAILURE. https://build.hpdd.intel.com//job/daos-stack/job/daos/view/change-requests/job/PR-12176/87/execution/node/327/log

daosbuild1 · 2023-06-06T13:25:01Z

Test stage Build DEB on Ubuntu 20.04 completed with status FAILURE. https://build.hpdd.intel.com//job/daos-stack/job/daos/view/change-requests/job/PR-12176/88/execution/node/245/log

daosbuild1 · 2023-06-06T13:27:44Z

Test stage Build RPM on EL 9 completed with status FAILURE. https://build.hpdd.intel.com//job/daos-stack/job/daos/view/change-requests/job/PR-12176/88/execution/node/248/log

daosbuild1 · 2023-06-06T13:28:42Z

Test stage Build RPM on EL 8 completed with status FAILURE. https://build.hpdd.intel.com//job/daos-stack/job/daos/view/change-requests/job/PR-12176/88/execution/node/242/log

daosbuild1 · 2023-06-06T13:30:10Z

Test stage Build RPM on Leap 15.4 completed with status FAILURE. https://build.hpdd.intel.com//job/daos-stack/job/daos/view/change-requests/job/PR-12176/88/execution/node/237/log

daosbuild1 · 2023-06-06T14:12:41Z

Test stage Functional on Leap 15.4 completed with status FAILURE. https://build.hpdd.intel.com//job/daos-stack/job/daos/view/change-requests/job/PR-12176/89/execution/node/538/log

daosbuild1 · 2023-06-06T14:30:59Z

Test stage Functional on EL 8 completed with status FAILURE. https://build.hpdd.intel.com//job/daos-stack/job/daos/view/change-requests/job/PR-12176/89/execution/node/616/log

daosbuild1 · 2023-06-06T14:38:03Z

Test stage Functional on EL 9 completed with status FAILURE. https://build.hpdd.intel.com//job/daos-stack/job/daos/view/change-requests/job/PR-12176/89/execution/node/600/log

daosbuild1 · 2023-06-06T16:06:19Z

Test stage Build DEB on Ubuntu 20.04 completed with status FAILURE. https://build.hpdd.intel.com//job/daos-stack/job/daos/view/change-requests/job/PR-12176/90/execution/node/220/log

daosbuild1 · 2023-06-06T16:08:33Z

Test stage Build RPM on EL 9 completed with status FAILURE. https://build.hpdd.intel.com//job/daos-stack/job/daos/view/change-requests/job/PR-12176/90/execution/node/225/log

daosbuild1 · 2023-06-06T16:10:46Z

Test stage Build RPM on EL 8 completed with status FAILURE. https://build.hpdd.intel.com//job/daos-stack/job/daos/view/change-requests/job/PR-12176/90/execution/node/226/log

daosbuild1 · 2023-06-06T16:12:50Z

Test stage Build RPM on Leap 15.4 completed with status FAILURE. https://build.hpdd.intel.com//job/daos-stack/job/daos/view/change-requests/job/PR-12176/90/execution/node/248/log

daosbuild1

LGTM. No errors found by checkpatch.

FYI: Errors found in lines not modified in the patch:

ci/rpm/build_success.sh:12:
(lint) ShellCheck can't follow non-constant source. Use a directive to specify location. [SC1090]

utils/rpms/packaging/rpm_chrootbuild:118:
(lint) Use "${array[@]}" (with quotes) to prevent whitespace problems. [SC2048]
(lint) Use "${array[@]}" (with quotes) to prevent whitespace problems. [SC2048]

utils/rpms/packaging/rpm_chrootbuild:119:
(lint) eval negates the benefit of arrays. Drop eval to preserve whitespace/symbols (or eval as string). [SC2294]

ashleypittman · 2023-07-06T14:26:51Z

utils/githooks/pre-commit.d/20-flake.sh

+if [ ! -f .flake8 ]; then
+ echo " No .flake8, skipping flake checks"
+ exit 0
+fi
+


Is this change intentional?

It is. I don't recall the specific issue though. I am trying to make the githooks more portable to more repositories and must have run into some issue with .flake8 being missing.

ashleypittman · 2023-07-06T14:27:20Z

utils/githooks/pre-commit.d/40-pylint.sh

+
+if [ -f utils/cq/daos_pylint.py ]; then
+ if [ "$TARGET" = "HEAD" ]; then
+ echo " Checking against HEAD"
+ git diff HEAD --name-only | ./utils/cq/daos_pylint.py --files-from-stdin
+ else
+ echo " Checking against branch ${TARGET}"
+ git diff "$TARGET"... --name-only | ./utils/cq/daos_pylint.py --files-from-stdin
+ fi


Same comment for here. On this branch the files tested for do exist.

Again, it is likely due to trying to use the same githooks in other repos.

ashleypittman · 2023-07-06T15:18:53Z

debian/changelog

+daos (2.5.100-7) unstable; urgency=medium
+
+ [ Brian J. Murrell ]
+ * NOOP change to keep in parity with RPM version


It's no a NOOP

ashleypittman · 2023-07-06T15:19:21Z

site_scons/env_modules.py

@@ -56,6 +56,7 @@ def _module_func(self, command, *arguments): # pylint: disable=no-self-use

 # pylint: disable=consider-using-with
 try:
+ print(f"Going to run {cmd}")


Debug left in?

No. I left that in intentionally. I run into issues with this whole code path enough times that I think it's helpful to have that here.

it will appear in the build output for users, although I suppose most other commands do anyway.

Indeed. I think that is good though. It's providing some transparency about how CI is setting up the environment for MPI. It helps users that want to replicate CI have more understanding of what they need to do.

This results in quite a bit of debug that seems to only add the order of possible commands from what ia already printed, e.g.

2023-07-06 01:43:11,496 parameters L0143 DEBUG| PARAMS (key=mpi_type, path=/run/job_manager/*, default=mpich) => 'mpich' 2023-07-06 01:43:11,497 output L0696 DEBUG| [stdout] Going to run ['/usr/share/lmod/lmod/libexec/lmod', 'python', 'is-loaded', 'mpi/mpich-x86_64'] 2023-07-06 01:43:11,497 output L0696 DEBUG| [stdout] 2023-07-06 01:43:11,613 output L0696 DEBUG| [stdout] Going to run ['/usr/share/lmod/lmod/libexec/lmod', 'python', 'is-loaded', 'gnu-mpich'] 2023-07-06 01:43:11,614 output L0696 DEBUG| [stdout] 2023-07-06 01:43:11,731 output L0696 DEBUG| [stdout] Going to run ['/usr/share/lmod/lmod/libexec/lmod', 'python', 'is-loaded', 'mpi/mlnx_openmpi-x86_64'] 2023-07-06 01:43:11,731 output L0696 DEBUG| [stdout] 2023-07-06 01:43:11,849 output L0696 DEBUG| [stdout] Going to run ['/usr/share/lmod/lmod/libexec/lmod', 'python', 'is-loaded', 'mpi/openmpi3-x86_64'] 2023-07-06 01:43:11,849 output L0696 DEBUG| [stdout] 2023-07-06 01:43:11,966 output L0696 DEBUG| [stdout] Going to run ['/usr/share/lmod/lmod/libexec/lmod', 'python', 'is-loaded', 'gnu-openmpi'] 2023-07-06 01:43:11,966 output L0696 DEBUG| [stdout] 2023-07-06 01:43:12,084 output L0696 DEBUG| [stdout] Going to run ['/usr/share/lmod/lmod/libexec/lmod', 'python', 'is-loaded', 'mpi/openmpi-x86_64'] 2023-07-06 01:43:12,084 output L0696 DEBUG| [stdout] 2023-07-06 01:43:12,201 output L0696 DEBUG| [stdout] Trying to load mpi/mpich-x86_64 2023-07-06 01:43:12,201 output L0696 DEBUG| [stdout] 2023-07-06 01:43:12,202 output L0696 DEBUG| [stdout] Going to run ['/usr/share/lmod/lmod/libexec/lmod', 'python', 'is-avail', 'mpi/mpich-x86_64'] 2023-07-06 01:43:12,202 output L0696 DEBUG| [stdout] 2023-07-06 01:43:12,321 output L0696 DEBUG| [stdout] Going to run ['/usr/share/lmod/lmod/libexec/lmod', 'python', 'load', 'mpi/mpich-x86_64'] 2023-07-06 01:43:12,321 output L0696 DEBUG| [stdout] 2023-07-06 01:43:12,447 output L0696 DEBUG| [stdout] Loaded mpi/mpich-x86_64 2023-07-06 01:43:12,447 output L0696 DEBUG| [stdout] 2023-07-06 01:43:12,448 parameters L0143 DEBUG| PARAMS (key=bind_to, path=/run/mpirun/*, default=None) => None

from https://build.hpdd.intel.com/job/daos-stack/job/daos/job/PR-12176/124/artifact/Functional%20Hardware%20Medium/ior/small.py/job.log.

I think it should be removed.

ashleypittman · 2023-07-06T15:19:29Z

site_scons/env_modules.py

@@ -107,6 +108,7 @@ def _mpi_module(self, mpi):
 self._module_func('unload', to_unload)

 for to_load in load:
+ print(f"Trying to load {to_load}")


Debug left in.

brianjmurrell · 2023-07-06T19:44:07Z

Gatekeeper: Please DO NOT land this. I have removed daos-gatekeeper from the reviewers as all of the test-tool and middle-ware RPM PRs will need to land also or there will be massive amounts of test failures on master landings and timed runs -- where the EL9 test stage will run (it's only disabled for PRs, like Leap 15 is currently).

Given the time left this week, I don't think I can get all of those other PRs landed in a single day to prevent those massive failures for the 4 weeks I will be out on sabbatical starting Monday.

Signed-off-by: Brian J. Murrell <brian.murrell@intel.com>

daosbuild1

LGTM. No errors found by checkpatch.

FYI: Errors found in lines not modified in the patch:

ci/rpm/build_success.sh:12:
(lint) ShellCheck can't follow non-constant source. Use a directive to specify location. [SC1090]

utils/rpms/packaging/rpm_chrootbuild:118:
(lint) Use "${array[@]}" (with quotes) to prevent whitespace problems. [SC2048]
(lint) Use "${array[@]}" (with quotes) to prevent whitespace problems. [SC2048]

utils/rpms/packaging/rpm_chrootbuild:119:
(lint) eval negates the benefit of arrays. Drop eval to preserve whitespace/symbols (or eval as string). [SC2294]

daosbuild1 · 2023-08-08T16:58:24Z

Test stage Functional on EL 8 completed with status FAILURE. https://build.hpdd.intel.com//job/daos-stack/job/daos/view/change-requests/job/PR-12176/125/execution/node/1111/log

brianjmurrell · 2023-08-08T21:58:58Z

@JohnMalmberg @daltonbohning I'm hoping I can get some quick and timely reviews on the master merge that I had to do this PR for simply daos.spec and debian/changelog conflict update. @daos-stack/daos-gatekeeper willing, I am hoping to get this landed as early as possible tomorrow morning so that I can start landing the other side of the stack (test tools and middleware), hopefully all within the day so that there are not massive failures due to them being missing in tomorrow night's daily run which will include EL9 as a tested platform.

phender · 2023-07-10T21:31:30Z

src/tests/ftest/process_core_files.py

 dnf_args.extend(
- ["--enablerepo=*-debuginfo", "libpmemobj", "python3", "openmpi", "gcc"])
+ ["libpmemobj", "python3", "openmpi", "gcc"])


Is "--enablerepo=*-debuginfo" not required with EL8 anymore or is this an accidental exclusion?

I think that was (always?) superfluous. Or perhaps some previous bug(s) that were preventing installing the necessary debug packages has been fixed. In any case, core file testing is one thing that I focused on in this PR for all platforms.

phender · 2023-07-10T21:48:24Z

site_scons/env_modules.py

@@ -56,6 +56,7 @@ def _module_func(self, command, *arguments): # pylint: disable=no-self-use

 # pylint: disable=consider-using-with
 try:
+ print(f"Going to run {cmd}")


This results in quite a bit of debug that seems to only add the order of possible commands from what ia already printed, e.g.

2023-07-06 01:43:11,496 parameters L0143 DEBUG| PARAMS (key=mpi_type, path=/run/job_manager/*, default=mpich) => 'mpich' 2023-07-06 01:43:11,497 output L0696 DEBUG| [stdout] Going to run ['/usr/share/lmod/lmod/libexec/lmod', 'python', 'is-loaded', 'mpi/mpich-x86_64'] 2023-07-06 01:43:11,497 output L0696 DEBUG| [stdout] 2023-07-06 01:43:11,613 output L0696 DEBUG| [stdout] Going to run ['/usr/share/lmod/lmod/libexec/lmod', 'python', 'is-loaded', 'gnu-mpich'] 2023-07-06 01:43:11,614 output L0696 DEBUG| [stdout] 2023-07-06 01:43:11,731 output L0696 DEBUG| [stdout] Going to run ['/usr/share/lmod/lmod/libexec/lmod', 'python', 'is-loaded', 'mpi/mlnx_openmpi-x86_64'] 2023-07-06 01:43:11,731 output L0696 DEBUG| [stdout] 2023-07-06 01:43:11,849 output L0696 DEBUG| [stdout] Going to run ['/usr/share/lmod/lmod/libexec/lmod', 'python', 'is-loaded', 'mpi/openmpi3-x86_64'] 2023-07-06 01:43:11,849 output L0696 DEBUG| [stdout] 2023-07-06 01:43:11,966 output L0696 DEBUG| [stdout] Going to run ['/usr/share/lmod/lmod/libexec/lmod', 'python', 'is-loaded', 'gnu-openmpi'] 2023-07-06 01:43:11,966 output L0696 DEBUG| [stdout] 2023-07-06 01:43:12,084 output L0696 DEBUG| [stdout] Going to run ['/usr/share/lmod/lmod/libexec/lmod', 'python', 'is-loaded', 'mpi/openmpi-x86_64'] 2023-07-06 01:43:12,084 output L0696 DEBUG| [stdout] 2023-07-06 01:43:12,201 output L0696 DEBUG| [stdout] Trying to load mpi/mpich-x86_64 2023-07-06 01:43:12,201 output L0696 DEBUG| [stdout] 2023-07-06 01:43:12,202 output L0696 DEBUG| [stdout] Going to run ['/usr/share/lmod/lmod/libexec/lmod', 'python', 'is-avail', 'mpi/mpich-x86_64'] 2023-07-06 01:43:12,202 output L0696 DEBUG| [stdout] 2023-07-06 01:43:12,321 output L0696 DEBUG| [stdout] Going to run ['/usr/share/lmod/lmod/libexec/lmod', 'python', 'load', 'mpi/mpich-x86_64'] 2023-07-06 01:43:12,321 output L0696 DEBUG| [stdout] 2023-07-06 01:43:12,447 output L0696 DEBUG| [stdout] Loaded mpi/mpich-x86_64 2023-07-06 01:43:12,447 output L0696 DEBUG| [stdout] 2023-07-06 01:43:12,448 parameters L0143 DEBUG| PARAMS (key=bind_to, path=/run/mpirun/*, default=None) => None

from https://build.hpdd.intel.com/job/daos-stack/job/daos/job/PR-12176/124/artifact/Functional%20Hardware%20Medium/ior/small.py/job.log.

I think it should be removed.

brianjmurrell · 2023-08-09T20:44:18Z

I don't seem to be able to reply to #12176 (comment) directly, so will have to create a new comment here.

This results in quite a bit of debug that seems to only add the order of possible commands from what ia already printed, e.g.
...

I think it should be removed.

I left it in for transparency in how to reproduce what CI does to get the modules loaded. I can remove it if you think it's too much and too little value.

brianjmurrell self-assigned this May 20, 2023

daosbuild1 previously requested changes May 20, 2023

View reviewed changes

ci/provisioning/post_provision_config_common_functions.sh Outdated Show resolved Hide resolved

brianjmurrell force-pushed the bmurrell/el9 branch from 08f0db2 to 4c9d7e7 Compare May 20, 2023 17:10

brianjmurrell force-pushed the bmurrell/el9 branch 2 times, most recently from e5b6fdc to 6b8819b Compare June 5, 2023 18:21

daosbuild1 previously requested changes Jun 5, 2023

View reviewed changes

daosbuild1 previously requested changes Jun 6, 2023

View reviewed changes

brianjmurrell force-pushed the bmurrell/el9 branch from b071309 to b7adb9f Compare June 6, 2023 16:01

daosbuild1 reviewed Jul 5, 2023

View reviewed changes

brianjmurrell requested review from JohnMalmberg and daltonbohning July 5, 2023 22:27

daltonbohning previously approved these changes Jul 5, 2023

View reviewed changes

brianjmurrell changed the title ~~Build on EL9~~ DAOS-13224 build: Build and test on EL9 Jul 6, 2023

brianjmurrell changed the title ~~DAOS-13224 build: Build and test on EL9~~ DAOS-13224 build test: Build and test on EL9 Jul 6, 2023

JohnMalmberg previously approved these changes Jul 6, 2023

View reviewed changes

brianjmurrell requested a review from a team July 6, 2023 14:02

brianjmurrell added the forced-landing The PR has known failures or has intentionally reduced testing, but should still be landed. label Jul 6, 2023

ashleypittman reviewed Jul 6, 2023

View reviewed changes

brianjmurrell requested review from a team and removed request for a team July 6, 2023 16:19

Merge branch 'master' into bmurrell/el9

86b4f52

Signed-off-by: Brian J. Murrell <brian.murrell@intel.com>

brianjmurrell dismissed stale reviews from JohnMalmberg and daltonbohning via 86b4f52 August 8, 2023 14:05

daosbuild1 reviewed Aug 8, 2023

View reviewed changes

brianjmurrell requested review from JohnMalmberg and daltonbohning August 8, 2023 21:56

daltonbohning approved these changes Aug 8, 2023

View reviewed changes

JohnMalmberg approved these changes Aug 9, 2023

View reviewed changes

phender reviewed Aug 9, 2023

View reviewed changes

jolivier23 merged commit 74b68b8 into master Aug 9, 2023
34 of 35 checks passed

jolivier23 deleted the bmurrell/el9 branch August 9, 2023 20:42

DAOS-13224 build test: Build and test on EL9 #12176

DAOS-13224 build test: Build and test on EL9 #12176

Conversation

brianjmurrell commented May 20, 2023 • edited Loading

Before requesting gatekeeper:

Gatekeeper:

github-actions bot commented May 20, 2023 • edited Loading

daosbuild1 left a comment

Choose a reason for hiding this comment

daosbuild1 commented Jun 5, 2023

daosbuild1 left a comment

Choose a reason for hiding this comment

daosbuild1 commented Jun 5, 2023

daosbuild1 commented Jun 6, 2023

daosbuild1 left a comment

Choose a reason for hiding this comment

daosbuild1 commented Jun 6, 2023

daosbuild1 left a comment

Choose a reason for hiding this comment

daosbuild1 commented Jun 6, 2023

daosbuild1 commented Jun 6, 2023

daosbuild1 commented Jun 6, 2023

daosbuild1 commented Jun 6, 2023

daosbuild1 commented Jun 6, 2023

daosbuild1 commented Jun 6, 2023

daosbuild1 commented Jun 6, 2023

daosbuild1 commented Jun 6, 2023

daosbuild1 commented Jun 6, 2023

daosbuild1 commented Jun 6, 2023

daosbuild1 commented Jun 6, 2023

daosbuild1 commented Jun 6, 2023

daosbuild1 commented Jun 6, 2023

daosbuild1 left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

brianjmurrell commented Jul 6, 2023

daosbuild1 left a comment

Choose a reason for hiding this comment

daosbuild1 commented Aug 8, 2023

brianjmurrell commented Aug 8, 2023

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

brianjmurrell commented Aug 9, 2023

brianjmurrell commented May 20, 2023 •

edited

Loading

github-actions bot commented May 20, 2023 •

edited

Loading