-
Notifications
You must be signed in to change notification settings - Fork 9
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
add regression workflow #131
Conversation
.github/workflows/regression.yml
Outdated
|
||
- name: Checkout master branch | ||
run: | | ||
git fetch origin master |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Could we use latest binary from release?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
That should be doable yes. It depends on what should be tested: the latest release vs the current new features (PR branch), or the latest features (possibly unreleased) against the new features (PR branch) 🤔
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Let's test against latest release if possible, what is in master might not always be clean.
Also I am thinking it may help the test execution time if we do not have to compile.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Changed in 3badd60, see a successful workflow run here: https://github.com/antbern/rusty-pullauta/actions/runs/10808286676
I also added the output as a job summary which can also be seen in the link above 🚀
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Do you mind sharing the LAZ file you used in your test, I'll host it somewhere...
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Sure, here is a link. It is from Lantmäteriet and provided without license (CC0) so should be fine to host publicly 💯 It's about 180Mb and covers a 2,5km² area.
Perhaps there are better files with more varied terrain etc. Should probably also add a test that processes multiple files at some point...
8b8c4f0
to
3badd60
Compare
"""With the exception of GITHUB_TOKEN, secrets are not passed to the runner when a workflow is triggered from a forked repository.""" Let's use a variable instead. |
An attempt to create a CI workflow that can do some kind of regression test and compare the generate output map from the PR branch with the output of the
master
branch 🚀Steps:
REGRESSION_LAZ_FILE_URL
secret. You will need to do something similar to make this work.Checkout themaster
branch.Compile and run the program on themaster
branch.imagemagick
but it has some memory limits when running in the GitHub runners, so in the end I ended up withpngcomp
which checks pixel-by-pixel and outputs some metrics.What do you think? 🤔