Benchmarking

Measuring rsonpath's performance is crucial for verifying the core III. Performance principle. It's an involved process that is somehow orthogonal to the actual production code and public API of rsonpath, therefore it's delegated to a separate project, rsonpath-benchmarks, included as a git submodule.

Setup

The git submodule is initialized and updated as part of repo setup, with just init. The submodule is located in /crates/rsonpath-benchmarks. The Cargo.toml of that crate points to a local rsonpath-lib, by default it is assumed to be in ../rsonpath-lib relative to the benchmark crate.

External dependencies

Because we include other JSONPath implementations in the benchmarks to run differential benches against them, there are additional dependencies outside of the usual Rust/Cargo ecosystem.

1. An appropriate C++ compiler is required for the cc crate to compile the JSONSki code.
2. JDK of version at least 8 is required and your JAVA_HOME environment variable must be set to its location.

On x86_64 Ubuntu the latter can be done by installing openjdk-17-jdk and exporting JAVA_HOME as /usr/lib/jvm/java-1.17.0-openjdk-amd64.

Running the benchmarks

Downloading and validating the datasets for benchmarks is done automatically, but do note that the datasets are stored locally and may take upwards of 10GB on your local drive.

The main benchmarks run only rsonpath and are used to identify performance improvements and regressions. Run them with just bench. You can also run all benchmarks we have with just bench-all, but be advised it will take a lot of time.