Fuzzing maintained?

[glyn]

I'm not particularly conversant with fuzzing in Rust although I've used it elsewhere. However, I wonder whether the fuzz directory is maintained or possibly out of date? For example, there is mention of a non-existent Uri method is_relative_reference().

[yescallop]

Thanks for pointing it out. I've just fixed it. I used to manually run all the fuzz targets from time to time to see if the code still compiles - it was a tedious task! Today I learned that I can use cargo +nightly fuzz check to check them all at once.

So, yes, the fuzz directory is still maintained. I should note that there are currently two fuzz targets (against_iref and against_uriparse) failing because of bugs in those libraries (timothee-haudebourg/iref#21 and sgodwincs/uriparse-rs#22). Fuzzing has been very effective in finding bugs both in my and others' libraries, and even in RFC 3986: the last three deviations listed here are all due to fuzzing.

[glyn]

Thanks. It's good to know that the directory is maintained and that fuzzing is effective.

In the past, I've used fuzzing to generate a corpus of tests which can then be checked in and run as regular tests (r.g. during CI). Re-running fuzzing would extend the corpus. That was with Go. I wonder if the same thing can be done with Rust fuzzing?

[yescallop]

I guess it will work similarly with Rust. cargo-fuzz is based on LLVM's libFuzzer. It has an option -runs that specifies the number of individual test runs. When I ran cargo +nightly fuzz run parse -- -runs=0, the fuzzer gave the following output and exited:

INFO: Running with entropic power schedule (0xFF, 100).
INFO: Seed: 3290742115
INFO: Loaded 1 modules   (11239 inline 8-bit counters): 11239 [0x58e09eed88a0, 0x58e09eedb487), 
INFO: Loaded 1 PC tables (11239 PCs): 11239 [0x58e09eedb488,0x58e09ef072f8), 
INFO:      293 files found in /home/ye/work/fluent-uri/fuzz/corpus/parse
INFO: -max_len is not provided; libFuzzer will not generate inputs larger than 4096 bytes
INFO: seed corpus: files: 293 min: 1b max: 528b total: 8406b rss: 45Mb
#294    INITED cov: 426 ft: 786 corp: 223/4301b exec/s: 0 rss: 50Mb
#294    DONE   cov: 426 ft: 786 corp: 223/4301b lim: 528 exec/s: 0 rss: 50Mb
Done 294 runs in 0 second(s)

After I modified the library code to introduce a bug, I ran the command again and it failed with an error. It should then be possible to let the CI runner fetch the corpus and run the fuzzer as a test. This looks promising. It would be nice to have it set up in the future.