Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Limit physics to jobs to use MAKE_JOBS=1 #986

Merged
merged 3 commits into from
Aug 7, 2023

Conversation

j-rivero
Copy link
Contributor

@j-rivero j-rivero commented Aug 4, 2023

Two different implementations:

  • Legacy code ignition.dsl 3336df7 ; harcoding the check and the instructions in all the places.
  • New code based on yaml in gazebo_libs e564c79: implement pre_setup_script_hook: as part of the spec to support the injecting of random setup scripts for the different libraries. Define the MAKE_JOBS=1 for physics.

Signed-off-by: Jose Luis Rivero <jrivero@osrfoundation.org>
Signed-off-by: Jose Luis Rivero <jrivero@osrfoundation.org>
@j-rivero
Copy link
Contributor Author

j-rivero commented Aug 4, 2023

Code is adding a blank line for some of the jobs that did not have the extra_cmd before. Cosmetical issue, not sure if I can resolve it nicely. I see the MAKE_JOBS defined in -jammy- jobs for Harmonic and all the old jobs related to physics.

diff -ur -I '.*<id>dashboard_portlet_.*</id>.*' /tmp/current_xml_configuration/gz-physics6-debbuilder.xml /tmp/pr_xml_configuration/gz-physics6-debbuilder.xml
--- /tmp/current_xml_configuration/gz-physics6-debbuilder.xml	2023-08-04 17:42:15.965604778 +0000
+++ /tmp/pr_xml_configuration/gz-physics6-debbuilder.xml	2023-08-04 17:40:44.777599723 +0000
@@ -148,7 +148,7 @@
 
               
               
-if [ $(uname -m) = "aarch64" ]; then export MAKE_JOBS=1; fi
+export MAKE_JOBS=1
               /bin/bash -x ./scripts/jenkins-scripts/docker/multidistribution-ignition-debbuild.bash
               </command>
         </hudson.tasks.Shell>
diff -ur -I '.*<id>dashboard_portlet_.*</id>.*' /tmp/current_xml_configuration/gz-physics7-debbuilder.xml /tmp/pr_xml_configuration/gz-physics7-debbuilder.xml
--- /tmp/current_xml_configuration/gz-physics7-debbuilder.xml	2023-08-04 17:42:18.561606172 +0000
+++ /tmp/pr_xml_configuration/gz-physics7-debbuilder.xml	2023-08-04 17:40:47.373599141 +0000
@@ -148,7 +148,7 @@
 
               
               
-if [ $(uname -m) = "aarch64" ]; then export MAKE_JOBS=1; fi
+export MAKE_JOBS=1
               /bin/bash -x ./scripts/jenkins-scripts/docker/multidistribution-ignition-debbuild.bash
               </command>
         </hudson.tasks.Shell>
diff -ur -I '.*<id>dashboard_portlet_.*</id>.*' /tmp/current_xml_configuration/gz_cmake-ci-pr_any-jammy-amd64.xml /tmp/pr_xml_configuration/gz_cmake-ci-pr_any-jammy-amd64.xml
--- /tmp/current_xml_configuration/gz_cmake-ci-pr_any-jammy-amd64.xml	2023-08-04 17:41:34.977586579 +0000
+++ /tmp/pr_xml_configuration/gz_cmake-ci-pr_any-jammy-amd64.xml	2023-08-04 17:40:03.945563358 +0000
@@ -96,6 +96,7 @@
 
 
 
+
 export BUILDING_SOFTWARE_DIRECTORY=gz-cmake
 export ARCH=amd64
 /bin/bash -xe ./scripts/jenkins-scripts/docker/gz_cmake-compilation.bash
diff -ur -I '.*<id>dashboard_portlet_.*</id>.*' /tmp/current_xml_configuration/gz_common-ci-pr_any-jammy-amd64.xml /tmp/pr_xml_configuration/gz_common-ci-pr_any-jammy-amd64.xml
--- /tmp/current_xml_configuration/gz_common-ci-pr_any-jammy-amd64.xml	2023-08-04 17:41:36.181586405 +0000
+++ /tmp/pr_xml_configuration/gz_common-ci-pr_any-jammy-amd64.xml	2023-08-04 17:40:05.161566529 +0000
@@ -96,6 +96,7 @@
 
 
 
+
 export BUILDING_SOFTWARE_DIRECTORY=gz-common
 export ARCH=amd64
 /bin/bash -xe ./scripts/jenkins-scripts/docker/gz_common-compilation.bash
diff -ur -I '.*<id>dashboard_portlet_.*</id>.*' /tmp/current_xml_configuration/gz_fuel_tools-ci-pr_any-jammy-amd64.xml /tmp/pr_xml_configuration/gz_fuel_tools-ci-pr_any-jammy-amd64.xml
--- /tmp/current_xml_configuration/gz_fuel_tools-ci-pr_any-jammy-amd64.xml	2023-08-04 17:41:37.133586267 +0000
+++ /tmp/pr_xml_configuration/gz_fuel_tools-ci-pr_any-jammy-amd64.xml	2023-08-04 17:40:06.117568853 +0000
@@ -96,6 +96,7 @@
 
 
 
+
 export BUILDING_SOFTWARE_DIRECTORY=gz-fuel-tools
 export ARCH=amd64
 /bin/bash -xe ./scripts/jenkins-scripts/docker/gz_fuel_tools-compilation.bash
diff -ur -I '.*<id>dashboard_portlet_.*</id>.*' /tmp/current_xml_configuration/gz_gui-ci-pr_any-jammy-amd64.xml /tmp/pr_xml_configuration/gz_gui-ci-pr_any-jammy-amd64.xml
--- /tmp/current_xml_configuration/gz_gui-ci-pr_any-jammy-amd64.xml	2023-08-04 17:41:37.605586199 +0000
+++ /tmp/pr_xml_configuration/gz_gui-ci-pr_any-jammy-amd64.xml	2023-08-04 17:40:06.589569940 +0000
@@ -96,6 +96,7 @@
 
 
 
+
 export BUILDING_SOFTWARE_DIRECTORY=gz-gui
 export ARCH=amd64
 /bin/bash -xe ./scripts/jenkins-scripts/docker/gz_gui-compilation.bash
diff -ur -I '.*<id>dashboard_portlet_.*</id>.*' /tmp/current_xml_configuration/gz_launch-ci-pr_any-jammy-amd64.xml /tmp/pr_xml_configuration/gz_launch-ci-pr_any-jammy-amd64.xml
--- /tmp/current_xml_configuration/gz_launch-ci-pr_any-jammy-amd64.xml	2023-08-04 17:41:38.573586060 +0000
+++ /tmp/pr_xml_configuration/gz_launch-ci-pr_any-jammy-amd64.xml	2023-08-04 17:40:07.545572143 +0000
@@ -96,6 +96,7 @@
 
 
 
+
 export BUILDING_SOFTWARE_DIRECTORY=gz-launch
 export ARCH=amd64
 /bin/bash -xe ./scripts/jenkins-scripts/docker/gz_launch-compilation.bash
diff -ur -I '.*<id>dashboard_portlet_.*</id>.*' /tmp/current_xml_configuration/gz_math-ci-pr_any-jammy-amd64.xml /tmp/pr_xml_configuration/gz_math-ci-pr_any-jammy-amd64.xml
--- /tmp/current_xml_configuration/gz_math-ci-pr_any-jammy-amd64.xml	2023-08-04 17:41:35.709586473 +0000
+++ /tmp/pr_xml_configuration/gz_math-ci-pr_any-jammy-amd64.xml	2023-08-04 17:40:04.681565277 +0000
@@ -96,6 +96,7 @@
 
 
 
+
 export BUILDING_SOFTWARE_DIRECTORY=gz-math
 export ARCH=amd64
 /bin/bash -xe ./scripts/jenkins-scripts/docker/gz_math-compilation.bash
diff -ur -I '.*<id>dashboard_portlet_.*</id>.*' /tmp/current_xml_configuration/gz_msgs-ci-pr_any-jammy-amd64.xml /tmp/pr_xml_configuration/gz_msgs-ci-pr_any-jammy-amd64.xml
--- /tmp/current_xml_configuration/gz_msgs-ci-pr_any-jammy-amd64.xml	2023-08-04 17:41:36.421586370 +0000
+++ /tmp/pr_xml_configuration/gz_msgs-ci-pr_any-jammy-amd64.xml	2023-08-04 17:40:05.401567155 +0000
@@ -96,6 +96,7 @@
 
 
 
+
 export BUILDING_SOFTWARE_DIRECTORY=gz-msgs
 export ARCH=amd64
 /bin/bash -xe ./scripts/jenkins-scripts/docker/gz_msgs-compilation.bash
diff -ur -I '.*<id>dashboard_portlet_.*</id>.*' /tmp/current_xml_configuration/gz_physics-ci-main-jammy-amd64.xml /tmp/pr_xml_configuration/gz_physics-ci-main-jammy-amd64.xml
--- /tmp/current_xml_configuration/gz_physics-ci-main-jammy-amd64.xml	2023-08-04 17:41:38.213586112 +0000
+++ /tmp/pr_xml_configuration/gz_physics-ci-main-jammy-amd64.xml	2023-08-04 17:40:07.189571323 +0000
@@ -75,7 +75,7 @@
             <command>#!/bin/bash -xe
 
 
-
+export MAKE_JOBS=1
 export BUILDING_EXTRA_CMAKE_PARAMS=""
 export BUILDING_EXTRA_MAKETEST_PARAMS=""
 export BUILDING_SOFTWARE_DIRECTORY="gz-physics"
diff -ur -I '.*<id>dashboard_portlet_.*</id>.*' /tmp/current_xml_configuration/gz_physics-ci-pr_any-jammy-amd64.xml /tmp/pr_xml_configuration/gz_physics-ci-pr_any-jammy-amd64.xml
--- /tmp/current_xml_configuration/gz_physics-ci-pr_any-jammy-amd64.xml	2023-08-04 17:41:38.081586131 +0000
+++ /tmp/pr_xml_configuration/gz_physics-ci-pr_any-jammy-amd64.xml	2023-08-04 17:40:07.069571046 +0000
@@ -95,6 +95,7 @@
 export DISTRO=jammy
 
 
+export MAKE_JOBS=1
 
 export BUILDING_SOFTWARE_DIRECTORY=gz-physics
 export ARCH=amd64
diff -ur -I '.*<id>dashboard_portlet_.*</id>.*' /tmp/current_xml_configuration/gz_plugin-ci-pr_any-jammy-amd64.xml /tmp/pr_xml_configuration/gz_plugin-ci-pr_any-jammy-amd64.xml
--- /tmp/current_xml_configuration/gz_plugin-ci-pr_any-jammy-amd64.xml	2023-08-04 17:41:35.945586439 +0000
+++ /tmp/pr_xml_configuration/gz_plugin-ci-pr_any-jammy-amd64.xml	2023-08-04 17:40:04.921565903 +0000
@@ -96,6 +96,7 @@
 
 
 
+
 export BUILDING_SOFTWARE_DIRECTORY=gz-plugin
 export ARCH=amd64
 /bin/bash -xe ./scripts/jenkins-scripts/docker/gz_plugin-compilation.bash
diff -ur -I '.*<id>dashboard_portlet_.*</id>.*' /tmp/current_xml_configuration/gz_rendering-ci-pr_any-jammy-amd64.xml /tmp/pr_xml_configuration/gz_rendering-ci-pr_any-jammy-amd64.xml
--- /tmp/current_xml_configuration/gz_rendering-ci-pr_any-jammy-amd64.xml	2023-08-04 17:41:36.657586336 +0000
+++ /tmp/pr_xml_configuration/gz_rendering-ci-pr_any-jammy-amd64.xml	2023-08-04 17:40:05.641567756 +0000
@@ -96,6 +96,7 @@
 
 
 
+
 export BUILDING_SOFTWARE_DIRECTORY=gz-rendering
 export ARCH=amd64
 /bin/bash -xe ./scripts/jenkins-scripts/docker/gz_rendering-compilation.bash
diff -ur -I '.*<id>dashboard_portlet_.*</id>.*' /tmp/current_xml_configuration/gz_sensors-ci-pr_any-jammy-amd64.xml /tmp/pr_xml_configuration/gz_sensors-ci-pr_any-jammy-amd64.xml
--- /tmp/current_xml_configuration/gz_sensors-ci-pr_any-jammy-amd64.xml	2023-08-04 17:41:37.841586165 +0000
+++ /tmp/pr_xml_configuration/gz_sensors-ci-pr_any-jammy-amd64.xml	2023-08-04 17:40:06.829570493 +0000
@@ -96,6 +96,7 @@
 
 
 
+
 export BUILDING_SOFTWARE_DIRECTORY=gz-sensors
 export ARCH=amd64
 /bin/bash -xe ./scripts/jenkins-scripts/docker/gz_sensors-compilation.bash
diff -ur -I '.*<id>dashboard_portlet_.*</id>.*' /tmp/current_xml_configuration/gz_sim-ci-pr_any-jammy-amd64.xml /tmp/pr_xml_configuration/gz_sim-ci-pr_any-jammy-amd64.xml
--- /tmp/current_xml_configuration/gz_sim-ci-pr_any-jammy-amd64.xml	2023-08-04 17:41:38.333586094 +0000
+++ /tmp/pr_xml_configuration/gz_sim-ci-pr_any-jammy-amd64.xml	2023-08-04 17:40:07.305571590 +0000
@@ -96,6 +96,7 @@
 
 
 
+
 export BUILDING_SOFTWARE_DIRECTORY=gz-sim
 export ARCH=amd64
 /bin/bash -xe ./scripts/jenkins-scripts/docker/gz_sim-compilation.bash
diff -ur -I '.*<id>dashboard_portlet_.*</id>.*' /tmp/current_xml_configuration/gz_tools-ci-pr_any-jammy-amd64.xml /tmp/pr_xml_configuration/gz_tools-ci-pr_any-jammy-amd64.xml
--- /tmp/current_xml_configuration/gz_tools-ci-pr_any-jammy-amd64.xml	2023-08-04 17:41:35.217586544 +0000
+++ /tmp/pr_xml_configuration/gz_tools-ci-pr_any-jammy-amd64.xml	2023-08-04 17:40:04.185563984 +0000
@@ -96,6 +96,7 @@
 
 
 
+
 export BUILDING_SOFTWARE_DIRECTORY=gz-tools
 export ARCH=amd64
 /bin/bash -xe ./scripts/jenkins-scripts/docker/gz_tools-compilation.bash
diff -ur -I '.*<id>dashboard_portlet_.*</id>.*' /tmp/current_xml_configuration/gz_transport-ci-pr_any-jammy-amd64.xml /tmp/pr_xml_configuration/gz_transport-ci-pr_any-jammy-amd64.xml
--- /tmp/current_xml_configuration/gz_transport-ci-pr_any-jammy-amd64.xml	2023-08-04 17:41:37.369586233 +0000
+++ /tmp/pr_xml_configuration/gz_transport-ci-pr_any-jammy-amd64.xml	2023-08-04 17:40:06.353569396 +0000
@@ -96,6 +96,7 @@
 
 
 
+
 export BUILDING_SOFTWARE_DIRECTORY=gz-transport
 export ARCH=amd64
 /bin/bash -xe ./scripts/jenkins-scripts/docker/gz_transport-compilation.bash
diff -ur -I '.*<id>dashboard_portlet_.*</id>.*' /tmp/current_xml_configuration/gz_utils-ci-pr_any-jammy-amd64.xml /tmp/pr_xml_configuration/gz_utils-ci-pr_any-jammy-amd64.xml
--- /tmp/current_xml_configuration/gz_utils-ci-pr_any-jammy-amd64.xml	2023-08-04 17:41:35.457586509 +0000
+++ /tmp/pr_xml_configuration/gz_utils-ci-pr_any-jammy-amd64.xml	2023-08-04 17:40:04.445564662 +0000
@@ -96,6 +96,7 @@
 
 
 
+
 export BUILDING_SOFTWARE_DIRECTORY=gz-utils
 export ARCH=amd64
 /bin/bash -xe ./scripts/jenkins-scripts/docker/gz_utils-compilation.bash
diff -ur -I '.*<id>dashboard_portlet_.*</id>.*' /tmp/current_xml_configuration/ign-physics2-debbuilder.xml /tmp/pr_xml_configuration/ign-physics2-debbuilder.xml
--- /tmp/current_xml_configuration/ign-physics2-debbuilder.xml	2023-08-04 17:42:15.729604651 +0000
+++ /tmp/pr_xml_configuration/ign-physics2-debbuilder.xml	2023-08-04 17:40:44.545599758 +0000
@@ -148,7 +148,7 @@
 
               
               
-if [ $(uname -m) = "aarch64" ]; then export MAKE_JOBS=1; fi
+export MAKE_JOBS=1
               /bin/bash -x ./scripts/jenkins-scripts/docker/multidistribution-ignition-debbuild.bash
               </command>
         </hudson.tasks.Shell>
diff -ur -I '.*<id>dashboard_portlet_.*</id>.*' /tmp/current_xml_configuration/ign-physics5-debbuilder.xml /tmp/pr_xml_configuration/ign-physics5-debbuilder.xml
--- /tmp/current_xml_configuration/ign-physics5-debbuilder.xml	2023-08-04 17:42:15.845604713 +0000
+++ /tmp/pr_xml_configuration/ign-physics5-debbuilder.xml	2023-08-04 17:40:44.661599740 +0000
@@ -148,7 +148,7 @@
 
               
               
-if [ $(uname -m) = "aarch64" ]; then export MAKE_JOBS=1; fi
+export MAKE_JOBS=1
               /bin/bash -x ./scripts/jenkins-scripts/docker/multidistribution-ignition-debbuild.bash
               </command>
         </hudson.tasks.Shell>
diff -ur -I '.*<id>dashboard_portlet_.*</id>.*' /tmp/current_xml_configuration/ignition_physics-abichecker-any_to_any-ubuntu_auto-amd64.xml /tmp/pr_xml_configuration/ignition_physics-abichecker-any_to_any-ubuntu_auto-amd64.xml
--- /tmp/current_xml_configuration/ignition_physics-abichecker-any_to_any-ubuntu_auto-amd64.xml	2023-08-04 17:41:55.209581024 +0000
+++ /tmp/pr_xml_configuration/ignition_physics-abichecker-any_to_any-ubuntu_auto-amd64.xml	2023-08-04 17:40:24.037598593 +0000
@@ -147,7 +147,7 @@
 
                 
                 
-if [ $(uname -m) = "aarch64" ]; then export MAKE_JOBS=1; fi
+export MAKE_JOBS=1
 
                 export ARCH=amd64
                 export DEST_BRANCH=${DEST_BRANCH:-$ghprbTargetBranch}
diff -ur -I '.*<id>dashboard_portlet_.*</id>.*' /tmp/current_xml_configuration/sdformat-ci-pr_any-jammy-amd64.xml /tmp/pr_xml_configuration/sdformat-ci-pr_any-jammy-amd64.xml
--- /tmp/current_xml_configuration/sdformat-ci-pr_any-jammy-amd64.xml	2023-08-04 17:41:36.897586301 +0000
+++ /tmp/pr_xml_configuration/sdformat-ci-pr_any-jammy-amd64.xml	2023-08-04 17:40:05.881568309 +0000
@@ -96,6 +96,7 @@
 
 
 
+
 export BUILDING_SOFTWARE_DIRECTORY=sdformat
 export ARCH=amd64
 /bin/bash -xe ./scripts/jenkins-scripts/docker/sdformat-compilation.bash

Copy link
Contributor

@azeey azeey left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM! Just one minor comment.

@@ -87,6 +87,8 @@ void generate_ci_job(gz_ci_job, lib_name, branch, ci_config,
{
def distro = ci_config.system.version
def arch = ci_config.system.arch
def pre_setup_script = ci_config.pre_setup_script_hook.get(lib_name)?.join('\n')
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Do we need to check if pre_setup_script_hook exists first?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Do we need to check if pre_setup_script_hook exists first?

good catch. I added a protection 1f51c2b

Signed-off-by: Jose Luis Rivero <jrivero@osrfoundation.org>
@j-rivero j-rivero merged commit 1edad16 into master Aug 7, 2023
1 check passed
@j-rivero j-rivero deleted the jrivero/restrict_physics_to_make_1 branch August 7, 2023 14:24
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants