…i-calls-adapters

Test parts of API with libFuzzer.

Closes oneapi-src#789

[UR] Remove `cuda_`/`hip_` prefixes from tests files.

[adapters] update to latest sycl (with 0.7.1 tag)

…graph (#10789)

This PR contains a set of changes that implement throwing an exception
when a feature unsupported by

[sycl_ext_oneapi_graph](https://github.com/intel/llvm/blob/sycl/sycl/doc/extensions/proposed/sycl_ext_oneapi_graph.asciidoc)
is used.

An error is thrown when an application uses the following features with
the graph extension:
* An unsupported extension.
* Querying the event returned from graph submission for profiling
information.
* Level Zero immediate command-list (see
intel/llvm#10467)
* Specialization constants, kernel bundles, or reductions are used in a
graph node.

## Authors

Co-authored-by: Pablo Reble <pablo.reble@intel.com>
Co-authored-by: Julian Miller <julian.miller@intel.com>
Co-authored-by: Ben Tracy <ben.tracy@codeplay.com>
Co-authored-by: Ewan Crawford <ewan@codeplay.com>
Co-authored-by: Maxime France-Pillois
<maxime.francepillois@codeplay.com>

Signed-off-by: Jaime Arteaga <jaime.a.arteaga.molina@intel.com>

Signed-off-by: Jaime Arteaga <jaime.a.arteaga.molina@intel.com>

This change is necessary to workaround a delightful bug in either HIP
runtime, or the HIP spec.

It's discussed at length in github.com/intel/llvm/issues/7252 but for
the purposes of this patch, it suffices to say that a call to
`hipMemPrefetchAsync` is *required* for correctness in the face of
global atomic operations on (*at least*) shared USM allocations.

The architecture of this change is slightly strange on first sight in
that we reduntantly track allocation information in several places. The
context now keeps track of all USM mappings. We require a mapping of
pointers to the allocated size, but these allocations aren't pinned to
any particular queue or HIP stream.
The `hipMemPrefetchAsync`, however, requires the associated HIP stream
object, and the size of the allocation. The stream comes
hot-off-the-queue *only* just before a kernel is launched, so we need to
defer the prefetch until we have that information.

Finally, the kernel itself keeps track of pointer arguments in a more
accessible way so we can determine which of the kernel's pointer
arguments do, in-fact, point to USM allocations.

In the CUDA/HIP adapters `urKernelSetArgValue` was being used to
implement both `urKernelSetArgValue` & `urKernelSetArgLocal`. However,
if the validation layer is enabled in UR then the path to set local arg
is never taken since it includes a check that `pArgValue` is not null.

This PR:
 * Implements `urKernelSetArgLocal` for CUDA/HIP adapters
* Changes `pi2ur` to call `urKernelSetArgLocal` when `arg_value` is
`nullptr`
* Implements `urKernelSetArgLocal` for L0 adapter - this just calls back
into `urKernelSetArgValue`.

In the CUDA/HIP adapters `urKernelSetArgValue` was being used to
implement both `urKernelSetArgValue` & `urKernelSetArgLocal`. However,
if the validation layer is enabled in UR then the path to set local arg
is never taken since it includes a check that `pArgValue` is not null.

This PR:
 * Implements `urKernelSetArgLocal` for CUDA/HIP adapters
* Changes `pi2ur` to call `urKernelSetArgLocal` when `arg_value` is
`nullptr`
* Implements `urKernelSetArgLocal` for L0 adapter - this just calls back
into `urKernelSetArgValue`.

In the HIP adapter the HIP module is not set until the program is built
with `urProgramBuild`, therefore we should check that the module
actually needs to be unloaded in `urProgramRelease`.

The following will result in a failure, but should still be valid UR
trace:

```cpp
uint8_t *source = "<some ptx>";
ur_program_handle_t prog;
urProgramCreate(context, device, sizeof(source), ptxSource, nullptr, &prog);
urProgramRelease(prog); // fails when it tries to unload the module.
```

Fix the license headers at the top of each source file in the unified
runtime directory.

---------

Co-authored-by: Alexey Bader <alexey.bader@intel.com>

Fix the license headers at the top of each source file in the unified
runtime directory.

---------

Co-authored-by: Alexey Bader <alexey.bader@intel.com>

While attempting to enable the Level-Zero and OpenCL adapters at the
same time git will report an error due to the OpenCL tag not existing in
the intel/llvm clone use for the Level-Zero adapter. Furthermore, if I
wanted to test separate bug fixes for the CUDA and HIP adapters from
different sources, this would also not be possible.

This patch uses a separate FetchSource for each adapter to allow each
adapters source to be fetched independently of the others and moves
those clones into the build directory. This does result in additional
git clones when enabling multiple adapters and again in multiple build
directories but avoids conflicting remote URL's or multiple build
configurations trampling on each others checked out tags.

and re-name the label to 'fuzz-short' for easier usage.

…e-clones

[cmake] Use separate FetchSource's per adapter

This change enables conformance tests on HW with adapters.
Tests that currently do not pass are ignored, but in the future,
when they are all fixed, tests should pass without using this functionality.

[cts] Add adapters ignore lists for conformance tests

Small updates in fuzz tests

This PR adds missing functions in the hip backend to allow for
interoperability in programs that create sycl objects from native hip
objects. The new function implementations are:

- `make_device`
- `make_queue`
- `make_event`

Note that it would really make sense for
intel/llvm#10491 to be merged first because this
PR makes the same code change in pi2ur, for a fix that is attributed to
#10491.

---------

Signed-off-by: Jack Kirk <jack.kirk@codeplay.com>

`urinfo` is a command-line tool for inspecting the current execution
environment:

```console
$ build/bin/urinfo --help
usage: build/bin/urinfo [-h] [-v] [-V]

This tool enumerates Unified Runtime layers, adapters, platforms, and
devices which are currently visible in the local execution environment.

options:
  -h, --help            show this help message and exit
  --version             show version number and exit
  -v, --verbose         print additional information
```

[ur] Introduce urinfo tool

Test urDeviceGetGlobalTimestampTest.SuccessSynchronizedTime
is marked as OPT because is flaky.

When UR is built with the UR_BUILD_ADAPTER_L0 flag, run fuzz tests
with level zero adapter force loaded.

Add instructions for generating new short fuzz test scenarios.

[CTS] fix cts device adapter L0 match file

…10995)

Don't insert
zeCommandListAppendWaitOnEvents/zeCommandListAppendSignalEvent for
queue::ext_oneapi_submit_barrier() if we have in-order queue and all
events in the waitlist are from the same queue.

Creating device binaries silently fails if SYCL isn't available.

Add information on generating new corpus files

Run fuzz test with L0 when available

Cron scheduled workflows are triggered on the default branch only,
which is the 'main' branch.

Remove phony nightly CI workflow [adapters branch]

In current form this job is redundant, since we run the same builds
on the same OS, but with hardware and tests' execution.

LLVM: llvm/llvm-project@ac2d265
SPIRV-LLVM-Translator: KhronosGroup/SPIRV-LLVM-Translator@8701279

Adds `hipDeviceAttributeManagedMemory` attribute check to verify the HIP
device supports managed memory in order to use the prefetch API as well
as checking the pointer to migrate is actually a managed allocation
(USM/SVM).

If the check fails we return early and set a warning message (with
`UR_RESULT_SUCCESS` as return code).

As a follow-up. we may need to detect system-allocated memory and either
continue with just a warning message or throw a more targeted error
message and exit.

[adapters] Shorten CI

[ci] fix running cts tests that require device binaries

Correction to make_queue functionality. During the transition from L0
Plugin to UR Adapter a line of code in the make queue function was
inadvertenty dropped. This change restores it and also modifies the
associated test to do more careful checking.

Lots of hip/cu driver API calls were wrapped in `ur::assertion(res ==
CU_SUCCESS)` etc which:

- Means that any native error messages returned from the affected driver
api calls were lost.
- Since these APIs report errors asynchronously, such that they are
thrown from the last API call rather than the call which led to the
error, previous asynchronous error messages from different APIs to the
ones wrapped by the `ur::assertion` could also be lost depending on user
code.

These problems are fixed by swapping these assertions with
`UR_CHECK_ERROR`.

Note that in the future UR may want to adjust `UR_CHECK_ERROR` so that
it throws `UR_RESULT_ERROR_ADAPTER_SPECIFIC` etc instead of using
`std::cerr` etc to report the error etc. But I think it makes sense to
still use `UR_CHECK_ERROR` to wrap driver API calls because it means
that the __LINE__, __FUNCTION__ etc info can be correctly passed to
native error reporting.

---------

Signed-off-by: JackAKirk <jack.kirk@codeplay.com>

Based on #10758

Co-authored-by: Ronan Keryell <ronan@keryell.fr>

…parameter 'Alignment' warning (#11154)

Fixes
https://github.com/intel/llvm/actions/runs/6154418223/job/16699759620

…xtGetInfo (#10929)

Modified the return value for UR_CONTEXT_INFO_USM_FILL2D_SUPPORT and
UR_CONTEXT_INFO_USM_MEMCPY_2D_SUPPORT to return uint8_t instead of
uint32_t to comply with unified-runtime conformance test.

Signed-off-by: Zhang, Winston <winston.zhang@intel.com>

…d (#11075)

Signed-off-by: Spruit, Neil R <neil.r.spruit@intel.com>

This also makes SharedReadOnly allocations tracking obsolete.

Post-commit error fix for: intel/llvm#10807

Fetch the latest revision of unified runtime:

Notable changes
* Some command buffer entry-points have been renamed, also 2 additions
of membufferFill and USMFill
* UrInit/urTearDown have been removed - replaced with loader only
versions.

Fetch the latest revision of unified runtime:

Notable changes
* Some command buffer entry-points have been renamed, also 2 additions
of membufferFill and USMFill
* UrInit/urTearDown have been removed - replaced with loader only
versions.

…-example

End-to-end code generation example

This change adds a new aspect for esimd, `ext_intel_esimd`, and
annotates the two fundamental esimd classes, `simd_obj_impl` and
`simd_view_impl` with the `uses_aspect` attribute.

`simd_obj_impl` is the base class of `simd` and `simd_mask` which are
the fundamental user-facing classes.
`simd_view_impl` is the base of only `simd_view`. `simd_obj_impl` is not
a base of `simd_view_impl`, but every `simd_view_impl` requires a
reference to a `simd` or `simd_mask` at construction time, so I am not
sure if we truly need to annotate `simd_view_impl`, but I added it to be
safe.

It also adds a new PI device info query,
`PI_EXT_INTEL_DEVICE_INFO_ESIMD_SUPPORT` that is used to query at
runtime if a device supports ESIMD. For UR-based plugins, we map that to
`UR_DEVICE_INFO_ESIMD_SUPPOR`.

The implementation simply returns false for cuda, hip and native_cpu.
For l0 and opencl, we check that the device is an intel gpu by querying
the device type is gpu and the vendor id is `0x8086`.
For ESIMD emulator we simply return true.

I would appreciate careful review on the plugin changes in particular,
as I am not an expert.

This change also updates the esimd spec to document the new aspect.

In a future change, I plan to use the new aspect to remove the
requirement for the `SYCL_ESIMD_FUNCTION` and `SYCL_ESIMD_KERNEL` macros
that set function attributes, but I am not doing that as part of this PR
as it requires more investigation.

---------

Signed-off-by: Sarnie, Nick <nick.sarnie@intel.com>

This change adds a new aspect for esimd, `ext_intel_esimd`, and
annotates the two fundamental esimd classes, `simd_obj_impl` and
`simd_view_impl` with the `uses_aspect` attribute.

`simd_obj_impl` is the base class of `simd` and `simd_mask` which are
the fundamental user-facing classes.
`simd_view_impl` is the base of only `simd_view`. `simd_obj_impl` is not
a base of `simd_view_impl`, but every `simd_view_impl` requires a
reference to a `simd` or `simd_mask` at construction time, so I am not
sure if we truly need to annotate `simd_view_impl`, but I added it to be
safe.

It also adds a new PI device info query,
`PI_EXT_INTEL_DEVICE_INFO_ESIMD_SUPPORT` that is used to query at
runtime if a device supports ESIMD. For UR-based plugins, we map that to
`UR_DEVICE_INFO_ESIMD_SUPPOR`.

The implementation simply returns false for cuda, hip and native_cpu.
For l0 and opencl, we check that the device is an intel gpu by querying
the device type is gpu and the vendor id is `0x8086`.
For ESIMD emulator we simply return true.

I would appreciate careful review on the plugin changes in particular,
as I am not an expert.

This change also updates the esimd spec to document the new aspect.

In a future change, I plan to use the new aspect to remove the
requirement for the `SYCL_ESIMD_FUNCTION` and `SYCL_ESIMD_KERNEL` macros
that set function attributes, but I am not doing that as part of this PR
as it requires more investigation.

---------

Signed-off-by: Sarnie, Nick <nick.sarnie@intel.com>

Reverts intel/llvm#11155. I'm almost sure this is what broke our CI for
the past few days.

Reverts intel/llvm#11155. I'm almost sure this is what broke our CI for
the past few days.

…rnelLaunchExp (#11232)

This patch targets to address two issues found in the testing:
- Test hangs: This patch serializes the graph submissions in the tests
to prevent hangs when submitting multiple graphs.
- Issue adding a kernel to the command buffer in
`urCommandBufferAppendKernelLaunchExp`.

---------

Co-authored-by: Maxime France-Pillois <maxime.francepillois@codeplay.com>

…1213)

Signed-off-by: smaslov <sergey.v.maslov@intel.com>

The main fix here is to enable prefetch functionality when the HIP
backend is built with rocm versions < 5.
The change to the prefix/mem_advise UR hip adapter APIs fixes four
e2e-tests that fail test-e2e only for HIP 4.x versions (These failures
don't come up in the CI since it tests using rocm 5.x).

The change to Tracing/image_printers.cpp is effectively re-XFailing this
test for rocm 4 only. rocm 4 doesn't support `hipCreateSurfaceObject`
which is called by this test. I'm not sure this legacy image
functionality (Which as I understand it will be replaced by bindless
images) is really working for rocm 5 in a meaningful way, since most
other legacy image e2e tests are XFAIL for hip. But this test can still
be useful for us for ROCM 5 compatibility testing. The XFAIL was
recently removed here
intel/llvm@745febe
which led to us finding the rocm 4 fail. I guess that the CI at some
point switched from testing rocm 4 to testing rocm 5, which meant this
test stopped failing in CI.

---------

Signed-off-by: Jack Kirk <jack.kirk@codeplay.com>

`UR_CHECK_ERROR` was designed to return `ur_result_t`, however in
practice it was guaranteed to only ever return `UR_RESULT_SUCCESS`, as
other paths would either terminate, abort or throw.

This in turns leads to poor quality/error prone code, as the codebase
was littered with:
* statements not checking the return value - depending on the compiler
generating a warning,
* extra check on the return which was only ever going to be true.

Some care was required, as the codebase has a habit of accumulating err
codes across branches, so depending on the use case the initial value of
`ur_result_t Result`s had to be set accordingly (now that
`UR_CHECK_ERROR` does not return).

Fixes post-commit build fails in
https://github.com/intel/llvm/actions/runs/6256835642/job/16988269333

Signed-off-by: smaslov <sergey.v.maslov@intel.com>

See issue oneapi-src#888

…teWindowsCherrypick

[ADAPTERS] Cherry-pick "Temporarily disable running check-generated target in windows CI."

When a test fails a match, cmakes print both the
input file and the match file, so that it's easy to see
why the failure happened and how to fix it. Unfortunately,
cmake was inserting 3 newlines between every line of both
input and match files, making things hard to read.

This patch fixes this by moving match and input files printing
to the python script.

Co-authored-by: Petr Vesely <petr.vesely@codeplay.com>

[test] improve match file error output

Signed-off-by: Jaime Arteaga <jaime.a.arteaga.molina@intel.com>