diff --git a/CONTRIBUTING.md b/CONTRIBUTING.md index 464ffdd461..d1551ffa0c 100644 --- a/CONTRIBUTING.md +++ b/CONTRIBUTING.md @@ -200,21 +200,27 @@ If you're on windows, consider downloading [git for windows](https://git-scm.com If you're using linux you can run the ./runtime/Path{space}of{space}Building.exe executable with wine. You will need to provide a valid wine path to the emmy lua debugger directory. - ## Testing +Pob uses two types of tests. Unit tests that test specifc calculations or mechanics and "build diff" tests that calculate the effect the changes have on a given list of builds. +While both can be ran locally it's recommended to use the provided docker image for both ease of use and to keep everyone on the same page. + +To run the unit tests run `docker compose up busted-tests`. This option uses the [Busted](https://olivinelabs.com/busted/) framework to test values returned by specific functions. For more details see the implementation of the tests in `spec/System/`. + +To run the build difference tests run `docker compose up busted-diff`. `busted-diff` busted diff first computes a list of builds found in `spec/builds.txt` and `spec/TestBuilds/` while checked out to the current working changes. Then it uses git to checkout the branch specified by `DEVREF` or `dev` by defualt and computes the same builds. Lastly it finds the differences in the outputs generated from both, does some post processing, and prints all of the information out. -PoB uses the [Busted](https://olivinelabs.com/busted/) framework to run its tests. Tests are stored under `spec/System` and run automatically when a PR is modified. -More tests can be added to this folder to test specific functionality, or new test builds can be added to ensure nothing changed that wasn't intended. +`busted-diff` supports the following options through environment variables: +- `DEVREF`: Defines the branch to use as the reference against which to compare builds computed with new changes. +- `HEADREF`: Defines the branch which contains the changes to be tested. -### Running tests -1. Install [LuaRocks](https://luarocks.org/) -2. Run `luarocks install busted` -3. Run `busted --lua=luajit` from the command line. You may need to add `luajit` to your PATH +The following variables are meant to be internal only and only ever used with the `-e` option of docker compose, but are worth documenting nevertheless: +- `CACHEDIR`: Defines the directory where the cache of computed builds using `DEVREF` should be stored. -Docker alternative: +#### Saving output +By defualt the output is not saved to a file but you can do so some your shell by either using `docker compose up busted-tests > log` (platform agnostic) or `docker compose up busted-tests | tee log` which will also allow you to see the output as it's being printed out (powershell/linux). -1. Install [Docker](https://www.docker.com/get-started) -2. Run `docker-compose up` from the command line +Additionally, the output from `busted-diff` can be quite noisy. If you'd like to only get a specific part of the output you can use to following: +- `sed -n '/Runtime comparison for/,/Savefile Diff for/{/Savefile Diff for/!p;}' log` to only print out runtime information or nothing if nothing interesting is to be shown. +- `sed -n '/Savefile Diff for/, $p' /tmp/dockerlog` to print out parsed output mismatch information as well as raw text diff of the generated build xml. ### Debugging tests When running tests with a docker container it is possible to use emmylua for debugging. Paste in the following right under `function launch:OnInit()` in `./src/Launch.lua`: @@ -225,7 +231,7 @@ local dbg = require("emmy_core") dbg.tcpListen("localhost", 9966) dbg.waitIDE() ``` -After running `docker-compose up` the code will wait at the `dbg.waitIDE()` line until a debugger is attached. This will allow stepping through any code that is internal to POB but will not work for busted related code. +After running `docker-compose up busted-tests` the code will wait at the `dbg.waitIDE()` line until a debugger is attached. This will allow stepping through any code that is internal to POB but will not work for busted related code. Note that this only works for unit tests described above. ## Path of Building development tutorials