[lapack][blas][cuda] Update host task impl to use enqueue_native_command

[JackAKirk]

Description

Update host task impl to use enqueue_native_command for blas/lapack using the cuda backend (cublas/cusolver). I did both backends in a single PR because the cusolver backend uses the cublas backend of oneMKL.

The sycl_ext_codeplay_enqueue_native_command extension reduces latency wrt the host_task for native library submissions, and allows integration with sycl task_graph / events. See https://github.com/intel/llvm/blob/sycl/sycl/doc/extensions/experimental/sycl_ext_codeplay_enqueue_native_command.asciidoc
for details.

This extension has already been shown to lead to considerable performance improvements for applications that call oneMKL, such as Gromacs for the oneMKL fft backend. We expect similar improvements for the lapack and blas backends implemented here.

I had to update the lapack tests because they previously relied on the synchronous behaviour of the native calls due to the fact we had to sync the native streams, since previously with standard host_task we are not able to integrate the native event into the sycl task_graph/ sycl::event.
I did not need to update the blas tests since they already take into account asynchronous behaviour.

Checklist

All Submissions

• Do all unit tests pass locally? Attach a log.

I've added a test for each backend for each of the possible codepaths:

• SYCL_EXT_ONEAPI_ENQUEUE_NATIVE_COMMAND is defined: "..native_command"
• SYCL_EXT_ONEAPI_ENQUEUE_NATIVE_COMMAND is not defined so use previous code path with standard host_task

test_main_blas_ct_host_task.txt
test_main_blas_rt_host_task.txt
test_main_lapack_rt_native_command.txt
test_main_lapack_ct_native_command.txt
test_main_lapack_ct_host_task.txt
test_main_lapack_rt_host_task.txt
test_main_blas_ct_res_native_command.txt
test_main_blas_rt_res_native_command.txt

[JackAKirk]

This is the only change that also affects the code logic when "SYCL_EXT_ONEAPI_ENQUEUE_NATIVE_COMMAND" is not defined. I removed it because it is a duplication of the calls to the SYNC macro that happens within lapack function implementations.

[Rbiessy]

LGTM, thanks for the PR

[JackAKirk]

@hdelan please review this when you are back.

[JackAKirk]

I have a small patch ready to update cublas backend just a little to implement missing GEMV_BATCH.
This I think puts cublas backend to a status where everything that maps directly between oneMKL and cublas APIs is supported to some degree (some types remain unimplemented, such as bfloat16/some mixed precisions already identified in the issues board etc).
Is it OK for me to add it here, to save the PR review overhead?
@Rbiessy what do you think?

[Rbiessy]

I would prefer to have a separate PR to make it clearer which commit implements what.

[JackAKirk]

I would prefer to have a separate PR to make it clearer which commit implements what.

Yeah OK, I'll be patient, thanks.