Inline native assembly?

[ethanc8]

I realize that it might be hard to do this in SPIR-V, but would it be possible to put native assembly (e.g. AMDGCN code) or a higher-level assembly accepted by the OpenCL compiler/runtime/driver (e.g. Mesa NIR, Intel GEN vISA) inline in the device CUDA/HIP code, and have it then be used when on that device?

[pjaaskel]

It would require adding an extension to SPIR-V to define the inline asm text snippets (similarly to LLVM IR perhaps) and a mechanism to choose the correct snippet according to the JIT target (with a generic SPIR-V fallback). Definitely a useful feature, perhaps someone is working on this already.

[pjaaskel]

Only SPIR-V for device-side code for now. There have been discussions supporting an even fatter binary allowing to embed optimized target binaries (in addition to SPIR-V), but no actions to my knowledge. Needing to use inline assembly for optimal performance or implementing a feature not supported by SPIR-V could justify this feature.

[ethanc8]

I see. Thanks!

[pjaaskel]

For the record, there's https://github.com/intel/llvm/blob/sycl/sycl/doc/design/spirv-extensions/SPV_INTEL_inline_assembly.asciidoc which could be useful, but doesn't solve the problem of making SPIR-V target-specific if there are inline asm snippets.

[ethanc8]

Are there opcodes to detect the target or to detect if an extension is available?

[pjaaskel]

It seems no, it basically matches the LLVM IR's inline asm instruction. So the selection logic is still needed for JIT-time portability.

[ethanc8]

Another idea I had -- it might be possible to support inline PTX by converting it to SPIR-V. I believe ZLUDA (an implementation of libcuda for AMD GPUs) does this internally.

[ethanc8]

Unfortunately, even though AMD gave the author permission to release ZLUDA 3, they have now claimed that that permission was invalid and forced the author to take down ZLUDA 3. ZLUDA 2 already had a PTX-to-SPIR-V translator, but I'm not sure if there were major improvements made during the ZLUDA 3 development.

[pjaaskel]

If we can efficiently translate the ISA/IL to SPIR-V then the motivation is to support inputting the inline asm, not implementing/optimizing functionality for which SPIR-V is not efficient. It's a bit different use case and it's supported by the recently announced proprietary project with comparable goals to chipStar, SCALE for retargeting to AMD. Indeed for this use case the ZLUDA's PTX2SPIR-V translation tool might be interesting. Hopefully it will mature and be upstreamed to LLVM and also will support AMD IL for the AMD-specific HIP apps out there.