Skip to content

Latest commit

 

History

History
151 lines (136 loc) · 7.55 KB

fuzzing-other-architectures-on-azure.md

File metadata and controls

151 lines (136 loc) · 7.55 KB
Raw

Fuzzing non-X86_64 targets on Azure

Fuzzing non-x86_64 targets using libFuzzer on Azure makes use of two Open Source capabilities:

  1. libFuzzer, which performs the fuzzing
  2. qemu-user, which provides user-space emulation.

TL/DR: check out our libfuzzer-aarch64-crosscompile example

Issues using qemu_user based LibFuzzer in OneFuzz

There are a few notable limitations when using qemu-user based libFuzzer targets in OneFuzz.

  1. Live reproduction of crashes does not work.
  2. The libfuzzer-coverage task does not work.
  3. Only Linux is supported at this time.
  4. Only aarch64 CPU emulation has been tested. PRs are welcome to support other architectures.
  5. Custom setup scripts are not supported, though you can provide your own sysroot using --sysroot.

As such, a libfuzzer qemu_user template is available, which only uses the libfuzzer_fuzz and libfuzzer_crash_report. As these issues are resolve, the template will be updated to include the additional tasks.

Example

Let's build a simple aarch64 target using GCC as a cross-compiler (See our example).

  1. Make sure you have QEMU and the appropriate cross compiler installed:
sudo apt update
sudo apt install -y qemu-user g++-aarch64-linux-gnu
  1. Check out the libFuzzer libraries from compiler-rt. Note, GCC requires the use of pc-guard for instrumentation, which was removed by the compiler-rt project. As such, we need an older version of the library:
git clone https://github.com/llvm-mirror/compiler-rt
(cd compiler-rt; git checkout daa6759576548a2f3825faddaa6811cabbfb45eb)
  1. Build the libFuzzer libraries without ASAN:
mkdir -p fuzz-libs
(cd fuzz-libs; aarch64-linux-gnu-g++ -c ../compiler-rt/lib/fuzzer/*.cpp)
  1. Build our target:
aarch64-linux-gnu-g++ -pthread -lasan -o fuzz.exe fuzz-libs/*.o fuzz.c -fsanitize=address -fsanitize-coverage=trace-pc
  1. Verify our target built correctly:
ASAN_OPTIONS=:detect_leaks=0 qemu-aarch64 -L /usr/aarch64-linux-gnu ./fuzz.exe -help=1

NOTE: LSAN does not work in qemu-user, so we need to disable that.

Now we're ready to deploy this target to OneFuzz. Note, if we have custom libraries or want to run on a different version of linux, we'll need to provide our own sysroot.

  1. Now we can fuzz! Execute our fuzz.exe with qemu-aarch64 our inputs directory:
ASAN_OPTIONS=:detect_leaks=0 qemu-aarch64 -L /usr/aarch64-linux-gnu ./fuzz.exe ./inputs

In a few seconds, you'll see output that looks something like this:

INFO: Seed: 113138795
INFO:        1 files found in ./inputs/                                                                                                                    INFO: -max_len is not provided; libFuzzer will not generate inputs larger than 4096 bytes
INFO: seed corpus: files: 1 min: 3b max: 3b total: 3b rss: 314Mb
#2      INITED cov: 5 ft: 5 corp: 1/3b lim: 4 exec/s: 0 rss: 317Mb
#3      NEW    cov: 5 ft: 9 corp: 2/6b lim: 4 exec/s: 0 rss: 317Mb L: 3/3 MS: 1 ChangeBit-
#4      NEW    cov: 5 ft: 13 corp: 3/9b lim: 4 exec/s: 0 rss: 317Mb L: 3/3 MS: 1 ChangeBit-
#8      NEW    cov: 10 ft: 21 corp: 4/13b lim: 4 exec/s: 0 rss: 317Mb L: 4/4 MS: 4 ShuffleBytes-ChangeBit-ChangeBinInt-CrossOver-
#9      NEW    cov: 10 ft: 26 corp: 5/17b lim: 4 exec/s: 0 rss: 317Mb L: 4/4 MS: 1 CopyPart-
#10     NEW    cov: 10 ft: 31 corp: 6/21b lim: 4 exec/s: 0 rss: 317Mb L: 4/4 MS: 1 ChangeBit-
#11     NEW    cov: 10 ft: 36 corp: 7/25b lim: 4 exec/s: 0 rss: 317Mb L: 4/4 MS: 1 ShuffleBytes-
#12     NEW    cov: 10 ft: 37 corp: 8/28b lim: 4 exec/s: 0 rss: 317Mb L: 3/4 MS: 1 EraseBytes-
#16     NEW    cov: 10 ft: 45 corp: 9/32b lim: 4 exec/s: 0 rss: 317Mb L: 4/4 MS: 4 CopyPart-CopyPart-CrossOver-ShuffleBytes-
#18     NEW    cov: 11 ft: 46 corp: 10/36b lim: 4 exec/s: 0 rss: 317Mb L: 4/4 MS: 2 CopyPart-ChangeBit-
#19     NEW    cov: 11 ft: 47 corp: 11/40b lim: 4 exec/s: 0 rss: 317Mb L: 4/4 MS: 1 ShuffleBytes-
#20     REDUCE cov: 11 ft: 47 corp: 11/39b lim: 4 exec/s: 0 rss: 317Mb L: 2/4 MS: 1 EraseBytes-
#30     NEW    cov: 11 ft: 52 corp: 12/43b lim: 4 exec/s: 0 rss: 317Mb L: 4/4 MS: 5 ChangeBit-CrossOver-CrossOver-CrossOver-CrossOver-
#32     NEW    cov: 11 ft: 56 corp: 13/45b lim: 4 exec/s: 0 rss: 317Mb L: 2/4 MS: 2 CrossOver-ChangeBit-
#38     REDUCE cov: 11 ft: 56 corp: 13/44b lim: 4 exec/s: 0 rss: 317Mb L: 1/4 MS: 1 EraseBytes-

Launching our example in OneFuzz

These commands launches the a qemu-user based libFuzzer job in OneFuzz. Note, we've added the arguments --wait_for_running --wait_for_files inputs such that we can monitor our job until we've seen at least one new input found via fuzzing.

TARGET_PROJECT=AARCH64
TARGET_NAME=Example
TARGET_BUILD=1
FUZZ_POOL=linux
onefuzz template libfuzzer qemu_user ${TARGET_PROJECT} ${TARGET_NAME} ${TARGET_BUILD} ${FUZZ_POOL} --wait_for_running --wait_for_files inputs

When we run this, we'll see output similar to:

WARNING:onefuzz:qemu_user jobs are a preview feature and may change in the future
INFO:onefuzz:creating job (runtime: 24 hours)
INFO:onefuzz:created job: fa5b7870-a51b-4f79-924f-2ef11a9830a0
INFO:onefuzz:using container: oft-setup-5346d5f33bc35c3d94cbc70f7815b85e
INFO:onefuzz:using container: oft-inputs-9c31136dc16a5aab8edf7666a614a285
INFO:onefuzz:using container: oft-crashes-9c31136dc16a5aab8edf7666a614a285
INFO:onefuzz:using container: oft-reports-9c31136dc16a5aab8edf7666a614a285
INFO:onefuzz:using container: oft-unique-reports-9c31136dc16a5aab8edf7666a614a285
INFO:onefuzz:using container: oft-no-repro-9c31136dc16a5aab8edf7666a614a285
INFO:onefuzz:uploading target exe `fuzz.exe`
INFO:onefuzz:uploading /tmp/tmp_9_f9kc3/setup.sh
INFO:onefuzz:uploading /tmp/tmp_9_f9kc3/fuzz.exe-wrapper.sh
INFO:onefuzz:creating libfuzzer_fuzz task
INFO:onefuzz:creating libfuzzer_crash_report task
INFO:onefuzz:done creating tasks
- waiting on: libfuzzer_crash_report:init, libfuzzer_fuzz:init
- waiting on: libfuzzer_crash_report:waiting, libfuzzer_fuzz:scheduled
| waiting on: libfuzzer_crash_report:waiting, libfuzzer_fuzz:setting_up
/ waiting on: libfuzzer_crash_report:waiting
| waiting on: libfuzzer_crash_report:scheduled
\ waiting on: libfuzzer_crash_report:setting_up
INFO:onefuzz:tasks started
\ waiting for new files: oft-inputs-9c31136dc16a5aab8edf7666a614a285
INFO:onefuzz:new files found
{
    "config": {
        "build": "1",
        "duration": 24,
        "name": "Example",
        "project": "AARCH64"
    },
    "end_time": "2021-02-27T16:41:24+00:00",
    "job_id": "fa5b7870-a51b-4f79-924f-2ef11a9830a0",
    "state": "enabled",
    "task_info": [
        {
            "state": "stopped",
            "task_id": "15cc52b9-b15b-4cf7-9fa7-669db67c8e0b",
            "type": "libfuzzer_fuzz"
        },
        {
            "state": "running",
            "task_id": "b3466396-7047-46f5-a58d-a21ada881e97",
            "type": "libfuzzer_crash_report"
        }
    ],
    "user_info": {
        "application_id": "e3b350d1-7863-4bd5-a4c0-83e6436c9c09",
        "object_id": "232a2ac6-f8fc-4eb3-b427-0c91bbab7eea",
        "upn": "example@contoso.com"
    }
}

See Also