[7.17](backport #38162) PoC: use pipeline generator

[mergify]

Proposed commit message

• WHAT: pipeline migration from Jenkins to Buildkite using a pipeline generator based on Python.
• WHY: takes the concepts from [CI] Pipeline 2.0 for monorepos #20104 but for Buildkite

Tasks

• Enable when conditions.
  • GitHub comment - for individual stages and groups
  • GitHub labels - for individual stages
  • Changeset
  • Branches
• Use catalog to generate the pipeline itself, rather than calling a python script on a pipeline.yaml file.
• Support draft PRs.

Checklist

• My code follows the style guidelines of this project
• I have commented my code, particularly in hard-to-understand areas
• I have made corresponding changes to the documentation
• I have made corresponding change to the default configuration files
• I have added tests that prove my fix is effective or that my feature works
• I have added an entry in CHANGELOG.next.asciidoc or CHANGELOG-developer.next.asciidoc.

Author's Checklist

• [ ]

How to test this PR locally

Related issues

Use cases

Screenshots

Logs

---

This is an automatic backport of pull request #38162 done by [Mergify](https://mergify.com).

[botelastic]

This pull request doesn't have a Team:<team> label.

[elasticmachine]

💚 Build Succeeded

the below badges are clickable and redirect to their specific view in the CI or DOCS

Expand to view the summary

Build stats

• Duration: 56 min 5 sec

❕ Flaky test report

No test was executed to be analysed.

🤖 GitHub comments

Expand to view the GitHub comments

To re-run your PR in the CI, just comment with:

• /test : Re-trigger the build.

• /package : Generate the packages and run the E2E tests.

• /beats-tester : Run the installation tests with beats-tester.

• run elasticsearch-ci/docs : Re-trigger the docs validation. (use unformatted text in the comment!)