Test cl_khr_command_buffer_mutable_dispatch

[EwanC]

Test plan for cl_khr_command_buffer_mutable_dispatch breaking down the areas which need to be covered.

1 Device Info Query

Verify the following queries can be performed successfully and return
defined values:

• CL_DEVICE_MUTABLE_DISPATCH_CAPABILITIES_KHR

EDIT - Done in #1651

2 Mutable-commands

2.1 Mutable-dispatch

For each property in cl_mutable_dispatch_properties_khr reported by device query.

• Create a command-buffer with CL_COMMAND_BUFFER_MUTABLE_KHR
• Record clCommandNDRangeKernelKHR returning a cl_mutable_command_khr object
• Enqueue command-buffer and verify
• Set a new value for the mutable value, and enqueue again, verify result
• Enqueue command-buffer again, without updating mutable-dispatch, and verify that new values have persisted.

• CL_MUTABLE_DISPATCH_GLOBAL_OFFSET_KHR Add global offset tests for cl_khr_command_buffer_mutable_dispatch. #1743
• CL_MUTABLE_DISPATCH_GLOBAL_SIZE_KHR Add global size tests for cl_khr_command_buffer_mutable_dispatch. #1744
• CL_MUTABLE_DISPATCH_LOCAL_SIZE_KHR Add local size tests for cl_khr_command_buffer_mutable_dispatch. #1745
• CL_MUTABLE_DISPATCH_ARGUMENTS_KHR for normal arguments and SVM. Checking variants of:
  • __global arguments Add arguments tests for cl_khr_command_buffer_mutable_dispatch. #1749
  • __local arguments Add arguments tests for cl_khr_command_buffer_mutable_dispatch. #1749
  • POD (plain-old-data) arguments Add arguments tests for cl_khr_command_buffer_mutable_dispatch. #1749
  • NULL arguments Add arguments tests for cl_khr_command_buffer_mutable_dispatch. #1749
  • image arguments Add image arguments tests for cl_khr_command_buffer_mutable_dispatch. #1750
• CL_MUTABLE_DISPATCH_EXEC_INFO_KHR Part of SVM test in Add arguments tests for cl_khr_command_buffer_mutable_dispatch. #1749
• Test modifying all supported mutable properties in each update Added command buffer test with all mutable parameters dispatch #1905

2.1.1 Multiple updates

• Test using clUpdateMutableCommandsKHR() in a way which updates more than one command-handle in the same call. Added new test to cover multiple commands dispatch in one enqueued update #1919
• Test calling clUpdateMutableCommandsKHR() several times on the same command-handle and make sure that the most recent value persists. Added test to cover overwritten update of mutable parameters #1922
• Test clUpdateMutableCommandsKHR() being called multiple times on the same command before an enqueue, but with different arguments. Verify that all the combined updates are made correctly. Added test to cover iterative mutable argument update #1925

2.1.2 Simultaneous use

• single queue simultaneous-use Add out of order tests for cl_khr_command_buffer_mutable_dispatch. #1746

If a device supports simultaneous-use

• Test updating mutable-dispatch objects of a command-buffer while a previous instance of the command-buffer is in flight.

2.1.3 cross-queue simultaneous-use

• cross-queue simultaneous-use

EDIT: #1912

If a device supports simultaneous-use

Test updating mutable-dispatch objects of a command-buffer while a previous instance of the command-buffer is in flight to a different queue. See 1.3.3 from the cl_khr_comand_buffer test plan, but changing one of the command-buffer enqueues to the original queue.

2.2 clGetMutableCommandInfoKHR

Test each of the queries in clGetMutableCommandInfoKHR():

• CL_MUTABLE_COMMAND_COMMAND_QUEUE_KHR
• CL_MUTABLE_COMMAND_COMMAND_BUFFER_KHR
• CL_MUTABLE_DISPATCH_PROPERTIES_ARRAY_KHR
• CL_MUTABLE_DISPATCH_KERNEL_KHR
• CL_MUTABLE_DISPATCH_DIMENSIONS_KHR
• CL_MUTABLE_DISPATCH_GLOBAL_WORK_OFFSET_KHR
• CL_MUTABLE_DISPATCH_GLOBAL_WORK_SIZE_KHR
• CL_MUTABLE_DISPATCH_LOCAL_WORK_SIZE_KHR
• CL_MUTABLE_COMMAND_COMMAND_TYPE_KHR

Edit - Done in #1651

[EwanC]

Now that the oneAPI Construction Kit is open source, i'll highlight that it has unit tests for this extension that could be used for inspiration to fill in the CTS gaps, although it focuses on kernel argument update. https://github.com/codeplaysoftware/oneapi-construction-kit/tree/main/source/cl/test/UnitCL/source/cl_khr_command_buffer_mutable_dispatch

In particular, to the plan above:

Test using clUpdateMutableCommandsKHR() in a way which updates more than one command-handle in the same call.

https://github.com/codeplaysoftware/oneapi-construction-kit/blob/main/source/cl/test/UnitCL/source/cl_khr_command_buffer_mutable_dispatch/clUpdateMutableCommandsKHR.cpp#L1119

Test calling clUpdateMutableCommandsKHR() several times on the same command-handle and make sure that the most recent value persists.

https://github.com/codeplaysoftware/oneapi-construction-kit/blob/main/source/cl/test/UnitCL/source/cl_khr_command_buffer_mutable_dispatch/clUpdateMutableCommandsKHR.cpp#L764

Test update being called multiple times on the same command before an enqueue, but with different arguments. Verify that all the combined updates are made.

https://github.com/codeplaysoftware/oneapi-construction-kit/blob/main/source/cl/test/UnitCL/source/cl_khr_command_buffer_mutable_dispatch/clUpdateMutableCommandsKHR.cpp#L674

[EwanC]

Noting that the mutable dispatch asserts from PR KhronosGroup/OpenCL-Docs#992 also need CTS testing as described in #1897

[shajder]

hello @EwanC, I am begining my work on finishing this issue and after initial review I am a bit concerned about points 2.1.2/2.1.3. Work plan does not mention anything about out-of-order command queues, meanwhile there are already two (?) tests added addressing 2.1.2 which uses out-of-order queue, I am guessing inspired by cl_khr_command_queue preceding tests. Do we want to leave it as is or get rid of out-of-order queues ? Do we want the new test addressing 2.1.3 to use out-of-order command queue ? Thanks

[EwanC]

Hi @shajder, that's great you are able to pick this up again. That's a good point about out-of-order queues, device support for that is conditional on CL_COMMAND_BUFFER_CAPABILITY_OUT_OF_ORDER_KHR but we still want to test devices which don't support that optional capability in the 2.1.2 / 2.1.3 scenarios. So could you add tests that only use in-order queues for these scenarios? Realize that doing 2.1.2 again for an in-order queue might involve some duplication of the original test unless it can be refactored to also support in-order.

Regarding getting rid of the out-of-order queues, using an out-of-order queue is a valid use of the extension so I'm a bit reluctant to reduce coverage from a valid test.

[shajder]

... So could you add tests that only use in-order queues for these scenarios? Realize that doing 2.1.2 again for an in-order queue might involve some duplication of the original test unless it can be refactored to also support in-order.

make sense, I will refactor code to handle all mentioned cases

Regarding getting rid of the out-of-order queues, using an out-of-order queue is a valid use of the extension so I'm a bit reluctant to reduce coverage from a valid test.

understood, I will keep out-of-order test cases along with new in-order ones

[shajder]

@EwanC one more concern, in tests mutable_dispatch_global_size/mutable_dispatch_local_size/mutable_dispatch_global_offset argument cl_ndrange_kernel_command_properties_khr is not passed to clCommandNDRangeKernelKHR. Is this a mistake? According to spec (eg. CL_MUTABLE_DISPATCH_GLOBAL_OFFSET_KHR):

... Determines whether the global_work_offset of kernel execution can be modified after recording. If set, the global_work_offset of the kernel execution can be changed with clUpdateMutableCommandsKHR using the cl_mutable_dispatch_config_khr field of the mutable_config parameter. Otherwise, the global_work_offset cannot be modified.

[EwanC]

@EwanC one more concern, in tests mutable_dispatch_global_size/mutable_dispatch_local_size/mutable_dispatch_global_offset argument cl_ndrange_kernel_command_properties_khr is not passed to clCommandNDRangeKernelKHR. Is this a mistake? According to spec (eg. CL_MUTABLE_DISPATCH_GLOBAL_OFFSET_KHR):

... Determines whether the global_work_offset of kernel execution can be modified after recording. If set, the global_work_offset of the kernel execution can be changed with clUpdateMutableCommandsKHR using the cl_mutable_dispatch_config_khr field of the mutable_config parameter. Otherwise, the global_work_offset cannot be modified.

Another good question, there is the language in the spec for cl_mutable_dispatch_fields_khr that

If CL_MUTABLE_DISPATCH_UPDATABLE_FIELDS_KHR is not specified then it defaults to the value returned by the CL_DEVICE_MUTABLE_DISPATCH_CAPABILITIES_KHR device query.

Therefore I think the tests are correct by that default wording since CL_DEVICE_MUTABLE_DISPATCH_CAPABILITIES_KHR is checked in Skip(), however it does also leave a gap in testing about whether implementation correctly handle cl_ndrange_kernel_command_properties_khr if it is passed here. So I would say we could do something like improve the mutable_dispatch_global_size/mutable_dispatch_local_size/mutable_dispatch_global_offset tests to have a second run (or use a second kernel) where the clCommandNDRangeKernelKHR kernel is created with cl_ndrange_kernel_command_properties_khr

[shajder]

Realize that doing 2.1.2 again for an in-order queue might involve some duplication of the original test unless it can be refactored to also support in-order.

Added #1912 along with two test cases:
-mutable dispatch with simultaneous use in in-order queue
-mutable dispatch with simultaneous use in cross queue scenario

[EwanC]

Closing this issue as all the items in the test plan have been covered with a merged CTS PR. New issues can be open to track any future CTS gaps in cl_khr_command_buffer_mutable_dispatch