[20542] Refactor Github CI sanitizer related jobs #4662

EduPonz · 2024-04-05T15:25:53Z

Description

This PR refactors the Github CI for Address and Thread Sanitizers. It removes the old workflows and adds a single reusable one that defines 4 jobs:

asan_fastdds_build
asan_fastdds_test (uses the build from asan_fastdds_build)
asan_discovery_server_test (uses the build from asan_fastdds_build)
tsan_fastdds_test

Furthermore, it adds both a sanitizers-ci.yaml and nightly-sanitizers-ci.yaml so that:

On PR review request and once a day, all job run. For the case of PRs, they only run if there are no conflicts and the no-test and skip-ci labels are not set.
Manual dispatching allows for individually selecting whether to run asan_fastdds_test, asan_discovery_server_test, and tsan_fastdds_test. This way it is possible to run a subset of the jobs defined in the workflow.
Removes address_sanitizers.yaml
Removes thread_sanitizer.yaml

@Mergifyio backport 2.13.x 2.10.x 2.6.x
IMPORTANT: Remove the nightly workflows from the backports, as they will not run anyways.

Contributor Checklist

Commit messages follow the project guidelines.
N/A: The code follows the style guidelines of this project.
N/A: Tests that thoroughly check the new feature have been added/Regression tests checking the bug and its fix have been added; the added tests pass locally
N/A: Any new/modified methods have been properly documented using Doxygen.
N/A: Any new configuration API has an equivalent XML API (with the corresponding XSD extension)
N/A: Changes are ABI compatible.
N/A: Changes are API compatible.
N/A: New feature has been added to the versions.md file (if applicable).
N/A: New feature has been documented/Current behavior is correctly described in the documentation.
Applicable backports have been included in the description.

Reviewer Checklist

The PR has a milestone assigned.
The title and description correctly express the PR's purpose.
Check contributor checklist is correct.
Check CI results: changes do not issue any warning.
Check CI results: failing tests are unrelated with the changes.

…tdds Signed-off-by: eduponz <eduardoponz@eprosima.com>

Signed-off-by: eduponz <eduardoponz@eprosima.com>

…ath is 10 Signed-off-by: eduponz <eduardoponz@eprosima.com>

Signed-off-by: eduponz <eduardoponz@eprosima.com>

EduPonz · 2024-04-06T13:09:56Z

@Mergifyio backport 2.13.x 2.10.x 2.6.x

mergify · 2024-04-06T13:10:01Z

backport 2.13.x 2.10.x 2.6.x

✅ Backports have been created

#4675 [20542] Refactor Github CI sanitizer related jobs (backport #4662) has been created for branch 2.13.x but encountered conflicts
#4676 [20542] Refactor Github CI sanitizer related jobs (backport #4662) has been created for branch 2.10.x but encountered conflicts
#4677 [20542] Refactor Github CI sanitizer related jobs (backport #4662) has been created for branch 2.6.x but encountered conflicts

… reporting Signed-off-by: eduponz <eduardoponz@eprosima.com>

.github/workflows/sanitizers-ci.yml

.github/workflows/nightly-sanitizers-ci.yml

.github/workflows/reusable-sanitizers-ci.yml

Signed-off-by: eduponz <eduardoponz@eprosima.com>

JesusPoderoso

LGTM with green CI

* Refs #20542: Add Fast DDS Sanitizer worflow with support for asan fastdds Signed-off-by: eduponz <eduardoponz@eprosima.com> * Refs #20542: Add asan_discovery_server_test job Signed-off-by: eduponz <eduardoponz@eprosima.com> * Refs #20542: Add tsan_fastdds_test job Signed-off-by: eduponz <eduardoponz@eprosima.com> * Refs #20542: Remove colcon_meta input as max inputs for workflow_dispath is 10 Signed-off-by: eduponz <eduardoponz@eprosima.com> * Refs #20542: Create a reusable workflow and a CI one for PRs or manual Signed-off-by: eduponz <eduardoponz@eprosima.com> * Refs #20542: Add nightly sanitizer CI for all supported branches Signed-off-by: eduponz <eduardoponz@eprosima.com> * Refs #20542: Change extension to .yml Signed-off-by: eduponz <eduardoponz@eprosima.com> * Refs #20542: Remove old workflows Signed-off-by: eduponz <eduardoponz@eprosima.com> * Refs #20542: Run sanitizers CI on PR review request Signed-off-by: eduponz <eduardoponz@eprosima.com> * Refs #20542: Exclude xfail tests in discovery server test run Signed-off-by: eduponz <eduardoponz@eprosima.com> * Refs #20542: Install all python packages needed for asan_fastdds_test reporting Signed-off-by: eduponz <eduardoponz@eprosima.com> * Refs #20542: Apply Jesús' suggestions Signed-off-by: eduponz <eduardoponz@eprosima.com> --------- Signed-off-by: eduponz <eduardoponz@eprosima.com> (cherry picked from commit ac0e737) # Conflicts: # .github/workflows/address-sanitizers.yaml # .github/workflows/thread-sanitizer.yaml

* Refs #20542: Add Fast DDS Sanitizer worflow with support for asan fastdds Signed-off-by: eduponz <eduardoponz@eprosima.com> * Refs #20542: Add asan_discovery_server_test job Signed-off-by: eduponz <eduardoponz@eprosima.com> * Refs #20542: Add tsan_fastdds_test job Signed-off-by: eduponz <eduardoponz@eprosima.com> * Refs #20542: Remove colcon_meta input as max inputs for workflow_dispath is 10 Signed-off-by: eduponz <eduardoponz@eprosima.com> * Refs #20542: Create a reusable workflow and a CI one for PRs or manual Signed-off-by: eduponz <eduardoponz@eprosima.com> * Refs #20542: Add nightly sanitizer CI for all supported branches Signed-off-by: eduponz <eduardoponz@eprosima.com> * Refs #20542: Change extension to .yml Signed-off-by: eduponz <eduardoponz@eprosima.com> * Refs #20542: Remove old workflows Signed-off-by: eduponz <eduardoponz@eprosima.com> * Refs #20542: Run sanitizers CI on PR review request Signed-off-by: eduponz <eduardoponz@eprosima.com> * Refs #20542: Exclude xfail tests in discovery server test run Signed-off-by: eduponz <eduardoponz@eprosima.com> * Refs #20542: Install all python packages needed for asan_fastdds_test reporting Signed-off-by: eduponz <eduardoponz@eprosima.com> * Refs #20542: Apply Jesús' suggestions Signed-off-by: eduponz <eduardoponz@eprosima.com> --------- Signed-off-by: eduponz <eduardoponz@eprosima.com> (cherry picked from commit ac0e737) # Conflicts: # .github/workflows/address-sanitizers.yaml # .github/workflows/config/asan_colcon.meta # .github/workflows/thread-sanitizer.yaml

* Refs #20542: Add Fast DDS Sanitizer worflow with support for asan fastdds Signed-off-by: eduponz <eduardoponz@eprosima.com> * Refs #20542: Add asan_discovery_server_test job Signed-off-by: eduponz <eduardoponz@eprosima.com> * Refs #20542: Add tsan_fastdds_test job Signed-off-by: eduponz <eduardoponz@eprosima.com> * Refs #20542: Remove colcon_meta input as max inputs for workflow_dispath is 10 Signed-off-by: eduponz <eduardoponz@eprosima.com> * Refs #20542: Create a reusable workflow and a CI one for PRs or manual Signed-off-by: eduponz <eduardoponz@eprosima.com> * Refs #20542: Add nightly sanitizer CI for all supported branches Signed-off-by: eduponz <eduardoponz@eprosima.com> * Refs #20542: Change extension to .yml Signed-off-by: eduponz <eduardoponz@eprosima.com> * Refs #20542: Remove old workflows Signed-off-by: eduponz <eduardoponz@eprosima.com> * Refs #20542: Run sanitizers CI on PR review request Signed-off-by: eduponz <eduardoponz@eprosima.com> * Refs #20542: Exclude xfail tests in discovery server test run Signed-off-by: eduponz <eduardoponz@eprosima.com> * Refs #20542: Install all python packages needed for asan_fastdds_test reporting Signed-off-by: eduponz <eduardoponz@eprosima.com> * Refs #20542: Apply Jesús' suggestions Signed-off-by: eduponz <eduardoponz@eprosima.com> --------- Signed-off-by: eduponz <eduardoponz@eprosima.com> (cherry picked from commit ac0e737)

* Refactor Github CI sanitizer related jobs (#4662) * Refs #20542: Add Fast DDS Sanitizer worflow with support for asan fastdds Signed-off-by: eduponz <eduardoponz@eprosima.com> * Refs #20542: Add asan_discovery_server_test job Signed-off-by: eduponz <eduardoponz@eprosima.com> * Refs #20542: Add tsan_fastdds_test job Signed-off-by: eduponz <eduardoponz@eprosima.com> * Refs #20542: Remove colcon_meta input as max inputs for workflow_dispath is 10 Signed-off-by: eduponz <eduardoponz@eprosima.com> * Refs #20542: Create a reusable workflow and a CI one for PRs or manual Signed-off-by: eduponz <eduardoponz@eprosima.com> * Refs #20542: Add nightly sanitizer CI for all supported branches Signed-off-by: eduponz <eduardoponz@eprosima.com> * Refs #20542: Change extension to .yml Signed-off-by: eduponz <eduardoponz@eprosima.com> * Refs #20542: Remove old workflows Signed-off-by: eduponz <eduardoponz@eprosima.com> * Refs #20542: Run sanitizers CI on PR review request Signed-off-by: eduponz <eduardoponz@eprosima.com> * Refs #20542: Exclude xfail tests in discovery server test run Signed-off-by: eduponz <eduardoponz@eprosima.com> * Refs #20542: Install all python packages needed for asan_fastdds_test reporting Signed-off-by: eduponz <eduardoponz@eprosima.com> * Refs #20542: Apply Jesús' suggestions Signed-off-by: eduponz <eduardoponz@eprosima.com> --------- Signed-off-by: eduponz <eduardoponz@eprosima.com> (cherry picked from commit ac0e737) # Conflicts: # .github/workflows/address-sanitizers.yaml # .github/workflows/config/asan_colcon.meta # .github/workflows/thread-sanitizer.yaml * Refs #20703: Fix conflicts Signed-off-by: Mario Dominguez <mariodominguez@eprosima.com> * Refs #20703: Remove sanitizer nightlies Signed-off-by: Mario Dominguez <mariodominguez@eprosima.com> * Refs #20703: Remove old sanitizer jobs Signed-off-by: Mario Dominguez <mariodominguez@eprosima.com> * Refs #20703: set default discovery server branch to v1.2.1 Signed-off-by: Mario Dominguez <mariodominguez@eprosima.com> * #20703: Address review changes Signed-off-by: Mario Dominguez <mariodominguez@eprosima.com> --------- Signed-off-by: Mario Dominguez <mariodominguez@eprosima.com> Co-authored-by: Eduardo Ponz Segrelles <eduardoponz@eprosima.com> Co-authored-by: Mario Dominguez <mariodominguez@eprosima.com>

* Refs #20542: Add Fast DDS Sanitizer worflow with support for asan fastdds Signed-off-by: eduponz <eduardoponz@eprosima.com> * Refs #20542: Add asan_discovery_server_test job Signed-off-by: eduponz <eduardoponz@eprosima.com> * Refs #20542: Add tsan_fastdds_test job Signed-off-by: eduponz <eduardoponz@eprosima.com> * Refs #20542: Remove colcon_meta input as max inputs for workflow_dispath is 10 Signed-off-by: eduponz <eduardoponz@eprosima.com> * Refs #20542: Create a reusable workflow and a CI one for PRs or manual Signed-off-by: eduponz <eduardoponz@eprosima.com> * Refs #20542: Add nightly sanitizer CI for all supported branches Signed-off-by: eduponz <eduardoponz@eprosima.com> * Refs #20542: Change extension to .yml Signed-off-by: eduponz <eduardoponz@eprosima.com> * Refs #20542: Remove old workflows Signed-off-by: eduponz <eduardoponz@eprosima.com> * Refs #20542: Run sanitizers CI on PR review request Signed-off-by: eduponz <eduardoponz@eprosima.com> * Refs #20542: Exclude xfail tests in discovery server test run Signed-off-by: eduponz <eduardoponz@eprosima.com> * Refs #20542: Install all python packages needed for asan_fastdds_test reporting Signed-off-by: eduponz <eduardoponz@eprosima.com> * Refs #20542: Apply Jesús' suggestions Signed-off-by: eduponz <eduardoponz@eprosima.com> --------- Signed-off-by: eduponz <eduardoponz@eprosima.com> (cherry picked from commit ac0e737) Co-authored-by: Eduardo Ponz Segrelles <eduardoponz@eprosima.com>

* Refactor Github CI sanitizer related jobs (#4662) * Refs #20542: Add Fast DDS Sanitizer worflow with support for asan fastdds Signed-off-by: eduponz <eduardoponz@eprosima.com> * Refs #20542: Add asan_discovery_server_test job Signed-off-by: eduponz <eduardoponz@eprosima.com> * Refs #20542: Add tsan_fastdds_test job Signed-off-by: eduponz <eduardoponz@eprosima.com> * Refs #20542: Remove colcon_meta input as max inputs for workflow_dispath is 10 Signed-off-by: eduponz <eduardoponz@eprosima.com> * Refs #20542: Create a reusable workflow and a CI one for PRs or manual Signed-off-by: eduponz <eduardoponz@eprosima.com> * Refs #20542: Add nightly sanitizer CI for all supported branches Signed-off-by: eduponz <eduardoponz@eprosima.com> * Refs #20542: Change extension to .yml Signed-off-by: eduponz <eduardoponz@eprosima.com> * Refs #20542: Remove old workflows Signed-off-by: eduponz <eduardoponz@eprosima.com> * Refs #20542: Run sanitizers CI on PR review request Signed-off-by: eduponz <eduardoponz@eprosima.com> * Refs #20542: Exclude xfail tests in discovery server test run Signed-off-by: eduponz <eduardoponz@eprosima.com> * Refs #20542: Install all python packages needed for asan_fastdds_test reporting Signed-off-by: eduponz <eduardoponz@eprosima.com> * Refs #20542: Apply Jesús' suggestions Signed-off-by: eduponz <eduardoponz@eprosima.com> --------- Signed-off-by: eduponz <eduardoponz@eprosima.com> (cherry picked from commit ac0e737) # Conflicts: # .github/workflows/address-sanitizers.yaml # .github/workflows/config/asan_colcon.meta # .github/workflows/thread-sanitizer.yaml * Fix conflicts Signed-off-by: JesusPoderoso <jesuspoderoso@eprosima.com> --------- Signed-off-by: JesusPoderoso <jesuspoderoso@eprosima.com> Co-authored-by: Eduardo Ponz Segrelles <eduardoponz@eprosima.com> Co-authored-by: JesusPoderoso <jesuspoderoso@eprosima.com>

EduPonz requested a review from JesusPoderoso April 5, 2024 15:26

EduPonz force-pushed the ci/refactor_sanitizers branch from a96f890 to 77c793e Compare April 5, 2024 15:28

EduPonz requested review from JesusPoderoso and removed request for JesusPoderoso April 5, 2024 15:28

EduPonz force-pushed the ci/refactor_sanitizers branch 3 times, most recently from 6427c49 to f0cca47 Compare April 5, 2024 15:52

github-actions bot added the ci-pending PR which CI is running label Apr 5, 2024

EduPonz force-pushed the ci/refactor_sanitizers branch 11 times, most recently from 2c2dc06 to 78b0e40 Compare April 6, 2024 07:41

EduPonz added this to the v2.14.1 milestone Apr 6, 2024

EduPonz force-pushed the ci/refactor_sanitizers branch 2 times, most recently from f2820b6 to c98b293 Compare April 6, 2024 08:24

EduPonz added 8 commits April 6, 2024 10:37

Refs #20542: Add Fast DDS Sanitizer worflow with support for asan fas…

3bab68e

…tdds Signed-off-by: eduponz <eduardoponz@eprosima.com>

Refs #20542: Add asan_discovery_server_test job

1dc6e0e

Signed-off-by: eduponz <eduardoponz@eprosima.com>

Refs #20542: Add tsan_fastdds_test job

488a084

Signed-off-by: eduponz <eduardoponz@eprosima.com>

Refs #20542: Remove colcon_meta input as max inputs for workflow_disp…

8fcbbfc

…ath is 10 Signed-off-by: eduponz <eduardoponz@eprosima.com>

Refs #20542: Create a reusable workflow and a CI one for PRs or manual

9aa92aa

Signed-off-by: eduponz <eduardoponz@eprosima.com>

Refs #20542: Add nightly sanitizer CI for all supported branches

3c5baf4

Signed-off-by: eduponz <eduardoponz@eprosima.com>

Refs #20542: Change extension to .yml

e0cfc6f

Signed-off-by: eduponz <eduardoponz@eprosima.com>

Refs #20542: Remove old workflows

52f33cd

Signed-off-by: eduponz <eduardoponz@eprosima.com>

EduPonz requested a review from JesusPoderoso April 6, 2024 08:41

EduPonz added the needs-review PR that is ready to be reviewed label Apr 6, 2024

EduPonz mentioned this pull request Apr 6, 2024

[20542] Nightly sanitizer job for Fast DDS supported branches #4524

Closed

7 tasks

Refs #20542: Exclude xfail tests in discovery server test run

111de04

Signed-off-by: eduponz <eduardoponz@eprosima.com>

EduPonz requested review from JesusPoderoso and removed request for JesusPoderoso April 6, 2024 12:22

Refs #20542: Install all python packages needed for asan_fastdds_test…

0023ad0

… reporting Signed-off-by: eduponz <eduardoponz@eprosima.com>

EduPonz requested review from JesusPoderoso and removed request for JesusPoderoso April 7, 2024 06:17

JesusPoderoso suggested changes Apr 8, 2024

View reviewed changes

JesusPoderoso removed the needs-review PR that is ready to be reviewed label Apr 8, 2024

Refs #20542: Apply Jesús' suggestions

85a3de9

Signed-off-by: eduponz <eduardoponz@eprosima.com>

JesusPoderoso self-requested a review April 8, 2024 13:13

JesusPoderoso approved these changes Apr 8, 2024

View reviewed changes

JesusPoderoso added ready-to-merge Ready to be merged. CI and changes have been reviewed and approved. and removed ci-pending PR which CI is running labels Apr 9, 2024

EduPonz merged commit ac0e737 into master Apr 10, 2024
7 of 12 checks passed

EduPonz deleted the ci/refactor_sanitizers branch April 10, 2024 05:11

This was referenced Apr 10, 2024

[20542] Refactor Github CI sanitizer related jobs (backport #4662) #4675

Merged

[20542] Refactor Github CI sanitizer related jobs (backport #4662) #4676

Merged

[20542] Refactor Github CI sanitizer related jobs (backport #4662) #4677

Merged

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

[20542] Refactor Github CI sanitizer related jobs #4662

[20542] Refactor Github CI sanitizer related jobs #4662

EduPonz commented Apr 5, 2024 •

edited

Loading

EduPonz commented Apr 6, 2024

mergify bot commented Apr 6, 2024 •

edited

Loading

JesusPoderoso left a comment

[20542] Refactor Github CI sanitizer related jobs #4662

[20542] Refactor Github CI sanitizer related jobs #4662

Conversation

EduPonz commented Apr 5, 2024 • edited Loading

Description

Contributor Checklist

Reviewer Checklist

EduPonz commented Apr 6, 2024

mergify bot commented Apr 6, 2024 • edited Loading

✅ Backports have been created

JesusPoderoso left a comment

Choose a reason for hiding this comment

EduPonz commented Apr 5, 2024 •

edited

Loading

mergify bot commented Apr 6, 2024 •

edited

Loading