[FR] Nightly CI that tests each installer in the repo on Windows / macOS / Linux

[ajeetdsouza]

GitHub Actions would be very useful to test if an installer has broken due to some code change or a new package release. A daily runner would be able to find these issues before our users do.

[coolaj86]

I like that idea a lot.

Here's what I know about GitHub Actions: https://github.com/coolaj86/explore-github-actions

I don't know how to specify different OSes or do email reporting.

What do you know?

[ajeetdsouza]

The runs-on key that you're using allows you to specify different OSes. Typically, one would go with windows-latest, macos-latest, and ubuntu-latest. You might be able to use a matrix to prevent code duplication.

You could probably write a script that executes all of the installers in the repository in sequence. If any of the installers fail, exit with an error code. This causes the workflow to fail, which will send an email to the owner of the repository.

[coolaj86]

I made some progress towards this in webinstall/webi-installers@e9c3b79

@ajeetdsouza would you like to draft a GitHub Action?

[coolaj86]

Also, would you like to be an authorized reviewer / maintainer?

[ajeetdsouza]

I made some progress towards this in e9c3b79

Is there a better way we can do this? Updating 3 lists of packages every time a new one is added seems avoidable. Perhaps we maintain a static list of exceptions and just run webi on all directory names in the repository?

Also, the rm command isn't strictly necessary, since we're only going to be running this on CI and inside Docker.

@ajeetdsouza would you like to draft a GitHub Action?

I'm having a rather hectic month, so I won't be able to do so any time soon. If you create one, I'd be happy to review it.

Also, would you like to be an authorized reviewer / maintainer?

Unfortunately, I don't think I can spare the time to do justice as a maintainer. Thanks, though, I really appreciate the offer! 😄