Auto triggering tests bases on planning

[fruch]

ability to trigger multiple jobs base on the version specified in their planning.

for example if I have multiple jobs marked as planned for RC1, we should have a place to trigger all of those jobs with scylla_version as example: 2024.1.rc1

[k0machi]

Per our discussion here's what we can implement for this:

• A new type called "Release Plan"
• The release plan will contain a set of tests and conditions for this release plan, such as:
  • Job that must be run
  • Which version should they be run with
• The new release plan page should show the plan coverage and allow user to trigger all jobs within said plan with the specified version
• All our current views (release & custom views) should support selecting a set of jobs and creating a release plan out of them
• The release plan is distinct from assignee plan as of now (However that part should also be looked at at some point, as it is somewhat brittle. We could potentially rework assignee plans into release plans)
• The release plan lifetime can be set to a specified date or can be indefinite

[fruch]

Per our discussion here's what we can implement for this:

• A new type called "Release Plan"
• The release plan will contain a set of tests and conditions for this release plan, such as:
  • Job that must be run
  • Which version should they be run with
• The new release plan page should show the plan coverage and allow user to trigger all jobs within said plan with the specified version
• All our current views (release & custom views) should support selecting a set of jobs and creating a release plan out of them
• The release plan is distinct from assignee plan as of now (However that part should also be looked at at some point, as it is somewhat brittle. We could potentially rework assignee plans into release plans)
• The release plan lifetime can be set to a specified date or can be indefinite

Not sure about binding a plan to a release

It would limit us if we start talking about plan that spans multiple release folders
Example, master duty that that would span on enterprise and OSS.

But then on the other han, copy a plan in that sense is somewhat irrelevant.

Maybe consider the ties to release optional

[k0machi]

Maybe consider the ties to release optional

Hmm, indeed, that's a good point. My current idea only lists jobs, so untying those plans from a release is easy - we can basically do them the same as we do views, and for backlinks, e.g. viewing all plans that contain a specific release (master for example) we can store affected releases on a plan.

[fruch]

one more example of planning happening on master, regardless of a release, was tablets and raft topology.
for every major feature we might have some plans we want to execute before a release.

and we might have plan for staging as well, thing of a very complex SCT PR, we need to run all of artifacts tests at the same time ? (like I need few days ago)

one more idea to think about when designing plans, is plan that clones jobs. (not a first priority)

1. I have a set of test (by names / locations on master)
2. I want to clone them from master all into a folder in staging (or generate them from SCT code)
3. execute all of them with similar parameters

as reference code I'm using for such things (it's just triggering, not the cloning):

    for case, scylla_repo in [
        #('artifacts-ubuntu2204-test', latest_list),
        #('artifacts-ubuntu2004-arm-test', latest_list),
        #('artifacts-centos8-test', latest_repo),
        #('artifacts-rocky9-test', latest_repo),
        ('artifacts-debian10-arm-test', latest_list),
        #('artifacts-amazon2-arm-test', latest_repo),
        #('artifacts-amazon2-test', latest_repo),
        #('artifacts-ubuntu2004-fips-test', latest_list),
        #('artifacts-ubuntu2004-arm-test', latest_list),
        #('artifacts-ubuntu2404-test', latest_list),
        #('artifacts-ubuntu2404-arm-test', latest_list),
        #('artifacts-amazon2023-arm-test', latest_repo),
        #('artifacts-amazon2023-test', latest_repo),
        #('artifacts-amazon2023-offline_install-nonroot-test', ''),
        #('artifacts-amazon2023-offline_install-test', latest_repo),
        #('artifacts-centos9-arm-test', latest_repo),
        #('artifacts-centos9-selinux-test', latest_repo),
        #('artifacts-centos9-test', latest_repo),
        #('artifacts-centos9-web-test', latest_repo),
        ('artifacts-ubuntu2204-test' ,latest_list)
    ]:
        job_name = f'{folder}/{case}'
        job = JenkinsJobTrigger(job_name)
        job.update_scm(**repo_details)
        job.trigger_last({
            "gce_project": "gcp-sct-project-1",
            "scylla_version": '',
            "scylla_repo": scylla_repo,
            "email_recipients": "fruch@scylladb.com",
            "nonroot_offline_install": "false",
            "unified_package": '',
            #"unified_package": 'https://downloads.scylladb.com/unstable/scylla/master/relocatable/2024-06-08T18:51:41Z/scylla-unified-6.1.0~dev-0.20240608.f4706be8a8c4.x86_64.tar.gz',
        })

[k0machi]

Hmm, some sort of smart cloner would be nice in such case, so that you could specify config_files param that would append new files to each job instead of having to edit them manually / apply same to all of them

[fruch]

Hmm, some sort of smart cloner would be nice in such case, so that you could specify config_files param that would append new files to each job instead of having to edit them manually / apply same to all of them

That idea can be extended in lots of ways.

But we should start with the basis of it