Skip to content

Testing New Tests

Jump to bottom
Pankaj Mistry edited this page Feb 9, 2024 · 1 revision

This page describes steps that should be taken to verify any new tests before submitting them for review. Some apply for all tests, others are API-specific.

Common

Compiler Warnings

VK-GL-CTS has zero-warning policy. You should resolve any warnings generated by compilers that you can test with.

It is recommended, if possible, to build code with:

  • Recent versions of GCC, Clang, and Microsoft Visual Studio
  • Debug and Release builds
    • At least GCC generates additional warnings when optimization passes are enabled
  • 32-bit and 64-bit builds
    • One typical pitfall is implicitly casting VkDeviceSize (always 64-bit type) to size_t which generates warnings only on 32-bit builds

Android build is highly recommended as it covers clang and both 32-bit and 64-bit builds.

Valgrind

Tests must not do any invalid memory accesses (both on CPU as well as on GPU) nor leak memory. Since running valgrind on an actual driver can be quite slow, you can give -DDEQP_TARGET=null to cmake when generating build files using to enable a dummy API implementation that bypasses driver. Most tests will fail when ran against the dummy driver, but no valgrind memory errors or leaks should be reported.

Vulkan

API Usage Validation

Vulkan CTS tests must not violate API usage rules, as that would result in undefined (and untestable) behavior. Validation layers can be used to check for API usage errors.

Vulkan CTS framework provides command-line option for enabling validation layers. Layers must be installed on the system first, for example by downloading and installing LunarG SDK.

To start tests with validation enabled, use --deqp-validation=enable command line option. When validation is enabled, default instance and device (vkt::Context::getDevice() etc.) are created with validation layers enabled, and debug callback is registered to store any messages.

If any errors are found, they will be included at the end of the test case log, and test result will be set to InternalError.

SPIR-V Validation

vk-build-programs provides a command line option --validate-spv (or just -v) that runs all SPIR-V binaries through SPIR-V validator. To validate SPIR-V binaries in a specific test group, run:

./vk-build-programs --validate-spv -n dEQP-VK.pipeline.cache.*

Validation results and summary will be written to standard console output.

Clone this wiki locally