Skip to content

Latest commit

 

History

History
132 lines (101 loc) · 3.15 KB

testing.md

File metadata and controls

132 lines (101 loc) · 3.15 KB
Raw

Testing

The tests in libvfio-user are organized into a number of suites

  • unit - old unit tests written in C
  • functional - old functional tests using binaries written in C
  • pyunit - modern tests written in python
  • style - code style checking (eg flake8, rstlint, etc)

Running meson test -C build runs all the suites:

meson build
meson test -C build

It is possible to be selective about which tests are run by including or excluding suites:

meson test -C build --suite=pyunit
meson test -C build --no-suite=pyunit

The unit and pyunit suites support optional execution under valgrind

meson test -C build --suite=unit --setup=valgrind
meson test -C build --suite=pyunit --setup=pyvalgrind

To run with ASAN enabled, pass the -Db_sanitize=address option to meson. Note, this is incompatible with enabling valgrind

meson build -Db_sanitize=address
meson test -C build

The .github/workflows/pull_request.sh script runs a sequence of builds in various configurations. This is invoked for all pull requests, but can be launched manually by contributors ahead of opening a pull request.

The master branch is run through Coverity when a new PR lands.

Coverage reports can be enabled via meson, provided gcovr is installed:

meson build -Db_coverage=true
meson test -C build
ninja -C build coverage

The resulting coverage report can be viewed with

firefox file:`pwd`/build/meson-logs/coveragereport/index.html

Debugging Test Errors

Sometimes debugging Valgrind errors on Python unit tests can be tricky. To run specific tests, pass their name on the command line to meson:

meson build
meson test -C build test_quiesce.py

For even better control , use something like the following:

LIBVFIO_SO_DIR='/root/src/libvfio-user/build/lib' pytest-3 test/py/test_quiesce.py

To print libvfio-user's log messages, append the --capture=tee-sys option.

To use GDB to debug libvfio-user code called from Python unit tests, add a time.sleep somewhere to give you time to attach to the process and set breakpoints, run the tests with

meson test -C build --timeout-multiplier=10000

to prevent the test process from being killed by meson, and attach to the python3 process in the usual way using

gdb --pid=<pid>

AFL++

You can run American Fuzzy Lop against libvfio-user. It's easiest to use the Docker container:

cd /path/to/libvfio-user/src
docker pull aflplusplus/aflplusplus
docker run -ti -v $(pwd):/src aflplusplus/aflplusplus

Set up and build:

apt update
apt-get -y install libjson-c-dev libcmocka-dev clang valgrind \
                   python3-pytest debianutils flake8 meson

cd /src
export AFL_LLVM_LAF_ALL=1
CC=afl-clang-fast meson build -Dtran-pipe=true
ninja -C build

mkdir inputs
# don't yet have a better starting point
echo "1" >inputs/start
mkdir outputs

The VFU_TRAN_PIPE is a special libvfio-user transport that reads from stdin instead of a socket, we'll use this with the sample server to do our fuzzing:

afl-fuzz -i inputs/ -o outputs/ -- ./build/dbg/samples/server pipe