Fuzzers synchronization

[GrosQuildu]

DeepState implements ensembled fuzzing, where each fuzzer share interesting test cases with other. But synchronizing fuzzers is a bit tricky and needs some research, as it's badly documented.

Current state is:

• AFL and libFuzzer synchronize as for now.
• Eclipser and Honggfuzz need to be restared to pull new seeds
• Angora should synchronize (like AFL), but require correctly named files

See PP #326 for some initial information about the synchronization abilities.

Check out tests/test_fuzzers_sync.py for basic synchronization test. It pushes crashing seed into AFL queue directory and waits until all fuzzers share the seed and find the crash.

Fix would require:

• Implement do_restart in each fuzzer that doesn't automatically synchronize. It should use some heuristic to decide when to do restart (in simplest case, after some constant time).
• Implement ensembler script (deepstate-ensemble) that fix sync_dir from time to time: removes redundant files (using afl-cmin approach) and renames them appropriately (id:000001...), so AFL and Angora are happy. Maybe it should also decide when to do restart (not each fuzzer in do_restart method).
• Research if libFuzzer require some filename convention to pull new test cases

[GrosQuildu]

Some notes about how synchronization/ensembler may look like:

1. We can run multiple deepstate-afl by hand (or by some script or some option in deepstate-ensembler) with common sync_dir. Then we should write deepstate-ensembler as a tool that, when called, do: fixes sync_dir, collects and prints statistics and eventually restarts fuzzers (a bit like afl-whatsup).

   • Cons:
     • resources heavy: for X fuzzing processes we have X fuzzer processes + X python processes (which may be not that bad/heavy)
     • hard to maintaint fuzzers as a whole: we would need to store some config file inside sync_dir so deepstate-ensembler know where to look for each fuzzer workspace (for statistics and to check if they are live). Also detection of stopped fuzzers etc would be hard.
   • Pros:
     • ease to add/remove fuzzers: just run/stop how many frontends you want

2. We can run one deepstate-ensembler which has list of frontends objects (python objects). So we have one python process that manages all fuzzers (which is cool), but we would need some interface to talk to it. I.e running ensembler on new target create ensembler daemon and spawns some config file. Then running it again with the same config file as an argument, we can get statistic, add/stop fuzzers etc.

   • Cons:
     • Harder to add/remove fuzzers: depends on how we implement ensembler API
     • Harder to implement?
   • Pros:
     • Less resources usage: only one python process
     • Easier to resume fuzzing: even if ensembler daemon is killed, there would be that global config file from which we can resume all the fuzzers
     • Easier to maintaint as a whole: end-user doesn't have to care about maintaing fuzzers, ensembler would do it for him, but he still has control if he wants

3. Current solution. Now (or rather some time ago) deepstate-ensembler is somehow mixture of above. Maintains list of frontend objects, but we can't do anything with it except killing everything and running again.

   • Cons:
     • bad for continuous fuzzing: one fuzzer throws exception, everything need to be restarted
     • we depend only on sync_dir, hard to reuse fuzzers' workspaces
   • Pros:
     • is ok for short-time tasks: easy to start fuzzing session