Engine correctness tests

All changes to the engine must be reflected in engine corretness tests, located in crates/rsonpath-lib/tests/engine_correctness_test.rs. These are end-to-end tests that input a query and a file and test the results. This doc explains:

• how to create small tests for specific features;
• how to add tests for larger JSONs and generate expected results.

Simple tests

Every change to the functionality of the engine should be accompanied with small tests showcasing the feature. For example, the tests for basic functionality include things like:

• empty JSON files;
• JSON files with only the root object;
• hand-crafted JSON files with easy to predict results;
• artificial JSON documents constructed to specifically test how selectors interact.

When introducing a selector, you should at minimum create a very basic test for that specific selector, as well as test its interactions with existing selectors. Any cases that you find during debugging that give erroneus results should be turned into a small test.

Large tests

We have a number of bigger JSON files that we run correctness tests on – the Wikidata tests. These are multimegabyte documents that return thousands of results on queries. Every significant change to the engine should be accompanied with tests on those datasets. It is, quite naturally, infeasible to manually perform the query on such a big JSON to get the expected result. For that we use jq – a well-established and well-tested JSON query engine, which can be found on https://stedolan.github.io/jq/ and installed directly from your favourite package manager. It doesn't implement JSONPath, but rather a custom syntax for queries. It requires some reading to get a grasp of it, but once done one can easily construct a jq query equivalent to a JSONPath one and run it on the document in question. Note that jq is not oriented towards performance, so queries may take a long time.

For example, assume we want to run the query $..en.value on ./data/wikidata/wikidata_person.json. The equivalent jq query is:

[..|.en?|.value?|values]|length

We create an array of results for a given query and then return its length (the pipe operator | is the intuitive piping operation, flowing outputs from the left as inputs to the right). The query is defined as: recurse, seeking the label en, followed by value, and then return all matched values. The ? operator means "if no such label is found return null", and the values filter gathers only non-null results. Quite a bit more complex than JSONPath.