Ensure all targets are tested

[johananl]

It seems like not all targets are tested by the CI. I assume that's the case because some targets have been broken for a long time and no one seemed to notice. Examples:

• Raw build target is broken #879
• QEMU Flatcar build is broken #828

In addition, Flatcar (and maybe additional distros) isn't tested by the CI.

We should ensure all supported platforms are tested by the CI to avoid regressions.

[codenrhoden]

This is always been a question of having the resources/infra to run tests on. vSphere tests are run on VMC (paid for by VMware), a public IaaS provider. Same as Azure (paid for by MSFT I believe). AWS tests have been stalled for well over a year for integration with boskos (accounts managed by K8s/CNCF). There is currently no infra to run qemu builders. OCI hasn't offered resources for their builder, neither has Digital Ocean.

QEMU builders would at least be easy -- all you need is a VM anywhere with qemu installed.

[johananl]

Thanks for adding context @codenrhoden. Makes sense. So I guess running QEMU VMs on Prow itself is out of the question then? 🙂
Anyway, in case I have bandwidth for improving the CI, I'll look into a solution for QEMU builds. IMO it would make a lot of sense to use QEMU for testing all distros and even consider making these tests blocking for PRs, since many distro-specific changes are provider agnostic and could be caught on QEMU.

[codenrhoden]

When the tests are run on Prow, they are running on containers. I don't know of any way to get VMs, and even then, I doubt it would be with root access with the ability to install qemu.

If we do get the AWS stuff sorted out, though, I could see having a CI script that uses the AWS CLI to spin up a VM, gets qemu on that, and then do the build there.

[johananl]

I happen to have some experience with running VMs inside containers on a CI system, however it's tricky and I expect to run into Prow resource constraints and/or permission problems. I think your suggestion about spinning up a VM with QEMU makes the most sense.

[invidian]

CC @kopiczko re: testing more targets on CI :)

[k8s-triage-robot]

The Kubernetes project currently lacks enough contributors to adequately respond to all issues and PRs.

This bot triages issues and PRs according to the following rules:

• After 90d of inactivity, lifecycle/stale is applied
• After 30d of inactivity since lifecycle/stale was applied, lifecycle/rotten is applied
• After 30d of inactivity since lifecycle/rotten was applied, the issue is closed

You can:

• Mark this issue or PR as fresh with /remove-lifecycle stale
• Mark this issue or PR as rotten with /lifecycle rotten
• Close this issue or PR with /close
• Offer to help out with Issue Triage

Please send feedback to sig-contributor-experience at kubernetes/community.

/lifecycle stale

[johananl]

/remove-lifecycle stale

[k8s-triage-robot]

The Kubernetes project currently lacks enough contributors to adequately respond to all issues and PRs.

This bot triages issues and PRs according to the following rules:

• After 90d of inactivity, lifecycle/stale is applied
• After 30d of inactivity since lifecycle/stale was applied, lifecycle/rotten is applied
• After 30d of inactivity since lifecycle/rotten was applied, the issue is closed

You can:

• Mark this issue or PR as fresh with /remove-lifecycle stale
• Mark this issue or PR as rotten with /lifecycle rotten
• Close this issue or PR with /close
• Offer to help out with Issue Triage

Please send feedback to sig-contributor-experience at kubernetes/community.

/lifecycle stale

[invidian]

/remove-lifecycle stale

[k8s-triage-robot]

The Kubernetes project currently lacks enough contributors to adequately respond to all issues.

This bot triages un-triaged issues according to the following rules:

• After 90d of inactivity, lifecycle/stale is applied
• After 30d of inactivity since lifecycle/stale was applied, lifecycle/rotten is applied
• After 30d of inactivity since lifecycle/rotten was applied, the issue is closed

You can:

• Mark this issue as fresh with /remove-lifecycle stale
• Close this issue with /close
• Offer to help out with Issue Triage

Please send feedback to sig-contributor-experience at kubernetes/community.

/lifecycle stale

[invidian]

/remove-lifecycle stale

[k8s-triage-robot]

The Kubernetes project currently lacks enough contributors to adequately respond to all issues.

This bot triages un-triaged issues according to the following rules:

• After 90d of inactivity, lifecycle/stale is applied
• After 30d of inactivity since lifecycle/stale was applied, lifecycle/rotten is applied
• After 30d of inactivity since lifecycle/rotten was applied, the issue is closed

You can:

• Mark this issue as fresh with /remove-lifecycle stale
• Close this issue with /close
• Offer to help out with Issue Triage

Please send feedback to sig-contributor-experience at kubernetes/community.

/lifecycle stale

[invidian]

/remove-lifecycle stale

[k8s-triage-robot]

The Kubernetes project currently lacks enough contributors to adequately respond to all issues.

This bot triages un-triaged issues according to the following rules:

• After 90d of inactivity, lifecycle/stale is applied
• After 30d of inactivity since lifecycle/stale was applied, lifecycle/rotten is applied
• After 30d of inactivity since lifecycle/rotten was applied, the issue is closed

You can:

• Mark this issue as fresh with /remove-lifecycle stale
• Close this issue with /close
• Offer to help out with Issue Triage

Please send feedback to sig-contributor-experience at kubernetes/community.

/lifecycle stale

[AverageMarcus]

/remove-lifecycle stale
/help

Yeah, we still want this. If anyone can help out with this in any way that would be fantastic!

[k8s-ci-robot]

@AverageMarcus:
This request has been marked as needing help from a contributor.

Guidelines

Please ensure that the issue body includes answers to the following questions:

• Why are we solving this issue?
• To address this issue, are there any code changes? If there are code changes, what needs to be done in the code and what places can the assignee treat as reference points?
• Does this issue have zero to low barrier of entry?
• How can the assignee reach out to you for help?

For more details on the requirements of such an issue, please see here and ensure that they are met.

If this request no longer meets these requirements, the label can be removed
by commenting with the /remove-help command.

In response to this:

/remove-lifecycle stale
/help

Yeah, we still want this. If anyone can help out with this in any way that would be fantastic!

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository.