Skip to content

Commit

Permalink
DOCS: update docs to better reflect current testing implementation
Browse files Browse the repository at this point in the history
  • Loading branch information
Paliak committed Jul 3, 2024
1 parent 0e635ab commit 768f136
Showing 1 changed file with 17 additions and 11 deletions.
28 changes: 17 additions & 11 deletions CONTRIBUTING.md
Original file line number Diff line number Diff line change
Expand Up @@ -200,21 +200,27 @@ If you're on windows, consider downloading [git for windows](https://git-scm.com
If you're using linux you can run the ./runtime/Path{space}of{space}Building.exe executable with wine. You will need to provide a valid wine path to the emmy lua debugger directory.
## Testing
Pob uses two types of tests. Unit tests that test specifc calculations or mechanics and "build diff" tests that calculate the effect the changes have on a given list of builds.

Check warning on line 204 in CONTRIBUTING.md

View workflow job for this annotation

GitHub Actions / spellcheck

Unknown word (specifc)
While both can be ran locally it's recommended to use the provided docker image for both ease of use and to keep everyone on the same page.
To run the unit tests run `docker compose up busted-tests`. This option uses the [Busted](https://olivinelabs.com/busted/) framework to test values returned by specific functions. For more details see the implementation of the tests in `spec/System/`.
To run the build difference tests run `docker compose up busted-diff`. `busted-diff` busted diff first computes a list of builds found in `spec/builds.txt` and `spec/TestBuilds/` while checked out to the current working changes. Then it uses git to checkout the branch specified by `DEVREF` or `dev` by defualt and computes the same builds. Lastly it finds the differences in the outputs generated from both, does some post processing, and prints all of the information out.

Check warning on line 209 in CONTRIBUTING.md

View workflow job for this annotation

GitHub Actions / spellcheck

Unknown word (DEVREF)

Check warning on line 209 in CONTRIBUTING.md

View workflow job for this annotation

GitHub Actions / spellcheck

Unknown word (defualt)
PoB uses the [Busted](https://olivinelabs.com/busted/) framework to run its tests. Tests are stored under `spec/System` and run automatically when a PR is modified.
More tests can be added to this folder to test specific functionality, or new test builds can be added to ensure nothing changed that wasn't intended.
`busted-diff` supports the following options through environment variables:
- `DEVREF`: Defines the branch to use as the reference against which to compare builds computed with new changes.

Check warning on line 212 in CONTRIBUTING.md

View workflow job for this annotation

GitHub Actions / spellcheck

Unknown word (DEVREF)
- `HEADREF`: Defines the branch which contains the changes to be tested.

Check warning on line 213 in CONTRIBUTING.md

View workflow job for this annotation

GitHub Actions / spellcheck

Unknown word (HEADREF)
### Running tests
1. Install [LuaRocks](https://luarocks.org/)
2. Run `luarocks install busted`
3. Run `busted --lua=luajit` from the command line. You may need to add `luajit` to your PATH
The following variables are meant to be internal only and only ever used with the `-e` option of docker compose, but are worth documenting nevertheless:
- `CACHEDIR`: Defines the directory where the cache of computed builds using `DEVREF` should be stored.

Check warning on line 216 in CONTRIBUTING.md

View workflow job for this annotation

GitHub Actions / spellcheck

Unknown word (DEVREF)
Docker alternative:
#### Saving output
By defualt the output is not saved to a file but you can do so some your shell by either using `docker compose up busted-tests > log` (platform agnostic) or `docker compose up busted-tests | tee log` which will also allow you to see the output as it's being printed out (powershell/linux).

Check warning on line 219 in CONTRIBUTING.md

View workflow job for this annotation

GitHub Actions / spellcheck

Unknown word (defualt)
1. Install [Docker](https://www.docker.com/get-started)
2. Run `docker-compose up` from the command line
Additionally, the output from `busted-diff` can be quite noisy. If you'd like to only get a specific part of the output you can use to following:
- `sed -n '/Runtime comparison for/,/Savefile Diff for/{/Savefile Diff for/!p;}' log` to only print out runtime information or nothing if nothing interesting is to be shown.
- `sed -n '/Savefile Diff for/, $p' /tmp/dockerlog` to print out parsed output mismatch information as well as raw text diff of the generated build xml.

Check warning on line 223 in CONTRIBUTING.md

View workflow job for this annotation

GitHub Actions / spellcheck

Unknown word (dockerlog)
### Debugging tests
When running tests with a docker container it is possible to use emmylua for debugging. Paste in the following right under `function launch:OnInit()` in `./src/Launch.lua`:
Expand All @@ -225,7 +231,7 @@ local dbg = require("emmy_core")
dbg.tcpListen("localhost", 9966)
dbg.waitIDE()
```
After running `docker-compose up` the code will wait at the `dbg.waitIDE()` line until a debugger is attached. This will allow stepping through any code that is internal to POB but will not work for busted related code.
After running `docker-compose up busted-tests` the code will wait at the `dbg.waitIDE()` line until a debugger is attached. This will allow stepping through any code that is internal to POB but will not work for busted related code. Note that this only works for unit tests described above.

## Path of Building development tutorials

Expand Down

0 comments on commit 768f136

Please sign in to comment.