Vulkan CTS status

See our README for the current Vulkan Conformance Test suite (CTS) coverage status. Incoming PRs are expected to include the changes to the coverage, making it easy to track history.

Hooking up CTS

Setting up the CTS requires the VK-GL-CTS project from khronos and the vulkan installable client driver (ICD) provided by the gfx-portability project. The Khronos test suite contains a number of test cases for a vulkan implementation, but also for OpenGL and its GLES friends. You can either look at the build instructions on their project page to install all the components or follow the instructions below for building just the Vulkan CTS.

Windows

This assumes installing both the CTS and portability to the same directory:

Clone the projects

C:\dev\test> git clone https://github.com/KhronosGroup/VK-GL-CTS
C:\dev\test> git clone https://github.com/gfx-rs/portability

Compile the portability ICD (replace --features= with backend to build against):

C:\dev\test> cargo build --manifest-path=portability\libportability-icd\Cargo.toml --features=dx12

Building the CTS. Note that the cmake commands are specific to windows. For Linux you should follow the cmake commands in the CTS build instructions.

C:\dev\test> cd VK-GL-CTS\external && python fetch_sources.py && cd ..\..\
C:\dev\test> cmake VK-GL-CTS\ -G "Visual Studio 15 2017 Win64" -BVK-GL-CTS\build
C:\dev\test> cmake --build VK-GL-CTS\build --config Release --target deqp-vk
C:\dev\test> cmake --build VK-GL-CTS\build --config Release --target execserver

Linux

Same as for windows, but the CTS requires a patch to load the portability ICD:

diff --git a/framework/platform/lnx/tcuLnxVulkanPlatform.cpp b/framework/platform/lnx/tcuLnxVulkanPlatform.cpp
index 02720f68..e96ccb08 100644
--- a/framework/platform/lnx/tcuLnxVulkanPlatform.cpp
+++ b/framework/platform/lnx/tcuLnxVulkanPlatform.cpp
@@ -180,7 +180,7 @@ class VulkanLibrary : public vk::Library
 {
 public:
        VulkanLibrary (void)
-               : m_library     ("libvulkan.so.1")
+               : m_library     ("libportability.so")
                , m_driver      (m_library)
        {
        }
diff --git a/framework/platform/surfaceless/tcuSurfacelessPlatform.cpp b/framework/platform/surfaceless/tcuSurfacelessPlatform.cpp
index f1fb2d51..85f4b371 100644
--- a/framework/platform/surfaceless/tcuSurfacelessPlatform.cpp
+++ b/framework/platform/surfaceless/tcuSurfacelessPlatform.cpp
@@ -89,7 +89,7 @@ class VulkanLibrary : public vk::Library
 {
 public:
        VulkanLibrary (void)
-               : m_library     ("libvulkan.so.1")
+               : m_library     ("libportability.so")
                , m_driver      (m_library)
        {
        }

Running the CTS

In order to get our ICD to be recognized as a vulkan implementation by the vulkan loader we need to set the VK_ICD_FILENAMES environment variable to point to a json file specifying the ICD. This file is located at portability/libportability-icd/portability_debug.json and tells the loader what api version it implements and where it can find the dll (this file looks for the portability ICD built in debug mode, just switch to release if needed).

NOTE: Remember that the CTS will not complain if the VK_ICD_FILENAMES is not set and will instead use the system's vulkan installation.

Cherry is a browser-based GUI client and HTTP server for managing test case execution and browsing the results. See Setting up Cherry for instructions on how to setting it up.

If you prefer to run the CTS through the commandline, assuming both CTS and gfx-portability are built, one can run the suite with the following:

Windows:

cd VK-GL-CTS/build/external/vulkancts/modules/vulkan
Release\deqp-vk.exe --deqp-caselist-file=C:\Dev\test\VK-GL-CTS\external\vulkancts\mustpass\1.0.2\vk-default.txt

Linux:

cd VK-GL-CTS/external/vulkancts/modules/vulkan
LD_LIBRARY_PATH=<portability>/target/debug/ ./deqp-vk --deqp-caselist-file=<vulkancts>/external/vulkancts/mustpass/1.0.2/vk-default.txt

View the results with:

cp <vulkancts>/doc/testlog-stylesheet/testlog.* .
python <vulkancts>/scripts/log/log_to_xml.py TestResults.qpa TestResults.xml
firefox TestResults.xml