Testing early access or snapshot versions

[johanvos]

I see lots of progress and PR's getting merged, which is great.
I think we need to do more EA-releases or snapshot releases, and especially make sure that those are getting tested. With the OpenJFX releases, we see that many developers download ea-releases.
In order to guarantee the quality of Scene Builder, we need more developers to test ea versions. Building EA versions is easy, getting developers to test them is another thing though. How can we achieve this?

[AlmasB]

Linking #458 and #446 for the technical underpinning, which is in progress.

As for devs, one option might be to advertise an open call for testers, who would like to contribute but do not have time to work on the code.

In addition, we need a robust and automated testbed, @Oliver-Loeffler is making progress on it.

[johanvos]

As for devs, one option might be to advertise an open call for testers, who would like to contribute but do not have time to work on the code.
That will be key indeed.

[jperedadnr]

Before asking for testers, it would probably be good to have a set of artifacts for them. And #458 is the way to go to produce snapshots. So maybe we need to merge it as is now, and test it in the process (this is something that we need to do anyway)?

[AlmasB]

It appears all of the changes are in the build or CI files, so I vote "yes" on this approach. Sanity check and merge. Then refine until we are happy with the result.

[jperedadnr]

Since @abhinayagarwal had some requests on #458, once he approves, we should integrate it and start the process.

[jperedadnr]

Since #458 is done now, @AlmasB how can we start testing the early access builds? How do we open it to testers?

[AlmasB]

I suppose the first thing to do is to ensure that with every commit to master, a new pre-release is generated but the link (currently https://github.com/gluonhq/scenebuilder/releases/tag/early-access) remains the same. I've tested the link visibility without being signed in, which is great -- anyone can click the link and download the updated package (I've also run the windows package and it installs fine). We can double check the early release process works as expected during the next PR merge.

Once the above is ready, some ideas we can use:

1. Add the link to README and Scene Builder website with text along the lines of "latest stable release is here and latest development build is here"
2. Add an open call for testers in README. Have a dedicated issue where community members can sign up to be "testers". Once they watch or @ that issue, any post to the issue from maintainers should notify them, meaning we will able to request testing of specific builds.
3. In cases where there is a major change that's been merged, we can post a social media call (using the same link) to specifically target testing of that change.

[jperedadnr]

Great, tested on Mac the snapshot as well, it works fine.
I'm okay with those bullet points.
At some point we could somehow enable the same mechanism to build PRs for testing before they get merged (to test major changes or new features before merging).

[AlmasB]

At some point we could somehow enable the same mechanism to build PRs for testing before they get merged (to test major changes or new features before merging).

This is a great idea! I'm sure we'll be able to adapt the new GH actions to fit what's needed once we identify the goal.

1

I believe we now have:

1. Stable release GH action and a corresponding page in Releases for new users to download
2. Early release GH action and a corresponding page in Releases for new users who wish to test early release versions (e.g. a bug fix or a new feature they have been waiting for)

In both of these cases I emphasised new users because of packages (dmg, msi, etc.) that come with these releases. For a tester (or a maintainer testing such a PR or early release), this becomes cumbersome to have to delete one and reinstall another and productivity can be hindered.

I think what we currently lack is the ability to update SB from the installation, rather than asking the user to reinstall SB. This is just an idea at this stage, but it could simplify testing overall.

From what I have seen, the installation of SB relies on one .jar file to actually run SB. So technically speaking I think it's possible to update SB without reinstalling it by just downloading the .jar file? If this observation is correct, then a new theoretical workflow could for testers / maintainers:

1. A installs SB for the first time and chooses "early release channel".
2. B adds a new feature and merges a new commit.
3. GH action produces "ea" release and, in addition to dmg, msi, .. packages, includes a .jar file, and possibly a build version number file.
4. SB notifies A (or A clicks update) and SB downloads the .jar file and restarts itself, so A can see and use B's new feature.

2

As for testing PRs before merging, the above could also apply. We can have a GH action that is manually triggered to build artifacts but not release them. I think we are able to access these artifacts directly in "Actions" tab.

Building on the idea from above, suppose SB allows the user to enter the .jar file location and that way we can "update" SB to test built PRs before merging.

[johanvos]

I downloaded the first 18 ea release (from https://github.com/gluonhq/scenebuilder/releases/tag/sb18-early-access) and at least the debian version installs and starts great. Excellent work!