Raytracing support

[teaglin]

Hi, I was curious if there is any support for ray tracing. I am interested in prototyping raytracing applications on macOS with Metal 2 and then scaling them up for production on Nvidia's RTX GPUs.

[cdavis5e]

Theoretically, it should be possible to support the VK_NV_ray_tracing extension on Mac, using the Metal Performance Shaders framework which uses a very similar technique to accelerate ray tracing. At this time, though, ray tracing isn't supported.

[billhollings]

If anyone wants to take a run at implementing this...it would be welcome.

[teaglin]

I might be interested, but I probably won't be free for a few weeks to a month for other priorities.

[Lichtso]

Why not go straight for the newest (vendor independent) VK_KHR_ray_tracing extension?
https://www.khronos.org/news/press/khronos-group-releases-vulkan-ray-tracing

[billhollings]

Why not go straight for the newest (vendor independent) VK_KHR_ray_tracing extension?

This is an older thread. But yes...we should now aim for the KHR version.

[Lichtso]

One of the dependencies of VK_KHR_ray_tracing is VK_KHR_buffer_device_address. Which requires the host to know the physical addresses (pointers) inside the device. However, it seems that Metal does not expose that information to the user.

[oscarbg]

Came here searching for VK_KHR_buffer_device_address references, as is a recommended extension for current VKD3D Proton 2.0..
let me point out that now raytracing on Metal (on iOS 14.0 and Mac 11.0) seems more similar to VK_KHR_ray_tracing having added support for raytracing shaders (intersection, etc..) (i.e. integrated into Metal shading language)..
seems this new support requires on Mac:
*RDNA GPU or later (no Vega or earlier support)
*Intel Skylake GPU or later
and on IOS: A13 or later..
recalling from memory so might be wrong..

[dictoon]

Any news on this?

[oscarbg]

@krupitskas so you will work on spirv cross raytracing metal support?
also may be a good idea to look at https://github.com/ConfettiFX/The-Forge crossplatform raytracing support they support DXR Metal and Vulkan Rt.

[oscarbg]

seeing AMD RDNA2 6xxx GPUs now supported on MacOS 11.4 betas.. would be nice if someone can test if Metal Performance Shaders Raytracing demos get some "huge" speedups on these GPUs:

https://developer.apple.com/documentation/metalperformanceshaders/metal_for_accelerating_ray_tracing
https://developer.apple.com/documentation/metalperformanceshaders/animating_and_denoising_a_raytraced_scene

this may indicate it's being mapped to new RDNA2 raytracing ISA..

anyway WWDC 21 is near and we (hope)surely will get some update on raytracing on Metal on MacOS..

[billhollings]

NVIDIA Vulkan sample KhronosGroup/Vulkan-Samples#274 both include use of Ray Queries

[kode54]

Metal Performance Shaders Raytracing is also available on Apple Silicon, with the added benefit of being able to run in a single rendering pass.

[zmarlon]

Since buffer device address now available in Metal 3, VK_KHR_buffer_device_address has been included in the latest version of MoltenVK. Now you should actually be able to implement VK_KHR_ray_tracing? Is there any information in this direction?

[cdavis5e]

Since buffer device address now available in Metal 3, VK_KHR_buffer_device_address has been included in the latest version of MoltenVK. Now you should actually be able to implement VK_KHR_ray_tracing? Is there any information in this direction?

I have plans to work on this over the next six months. (Full disclosure: I'm being paid to do this.)

[natevm]

@cdavis5e I'm working on a Vulkan ray tracing library which uses most all features of Vulkan RT (SBT record data, AABB trees, etc). I'd be very interested to build off your work if I can. :)

[fknfilewalker]

What is the status on this?

[LeeTeng2001]

Also curious about the status

[mannewalis]

Any updates on the status of ray tracing support on Mac with MoltenVK?

[natevm]

Since buffer device address now available in Metal 3, VK_KHR_buffer_device_address has been included in the latest version of MoltenVK. Now you should actually be able to implement VK_KHR_ray_tracing? Is there any information in this direction?

I have plans to work on this over the next six months. (Full disclosure: I'm being paid to do this.)

Seeing as it’s been nearly a year since this comment was made, it might be worth hunting for another developer to work on this.

[AntarticCoder]

To implement raytracing, we need the dependencyVK_KHR_acceleration_structure first, and that extension needs VK_KHR_deferred_host_operations. If we were to implement acceleration structures, we could also implement VK_KHR_ray_query, so that could be some what easier.

I want to try a hand at implementing it, but I have no idea where to start, could someone give me some pointers. Thanks

Edit: I meant VK_KHR_deferred_host_operations not VK_KHR_deferred_operations

[natevm]

That would be a great start. VK_KHR_ray_query would also let you test things out without thinking about shader binding table details right away. Ultimately we'd want both queries and the full pipeline for portability reasons, but it's a good idea to break it all down.

I've never actually heard of VK_KHR_deferred_operations before. I see it's mentioned as VK_KHR_deferred_operation in some Intel RT slides, but other than that it's a bit of a dead end... Is this related to VK_KHR_deferred_host_operations? I seem to get more results for that.

https://registry.khronos.org/vulkan/specs/1.3-extensions/man/html/VK_KHR_deferred_host_operations.html
I've talked to Josh Barczak from that man page before on twitter, he's nice and iirc open to DMs. You might reach out to him @JoshuaBarczak

[AntarticCoder]

Thanks for the info, I'll try to contact @JoshuaBarczak.

[cdavis5e]

@gpx1000 ^^

[devshgraphicsprogramming]

To implement raytracing, we need the dependencyVK_KHR_acceleration_structure first, and that extension needs VK_KHR_deferred_host_operations. If we were to implement acceleration structures, we could also implement VK_KHR_ray_query, so that could be some what easier.

I want to try a hand at implementing it, but I have no idea where to start, could someone give me some pointers. Thanks

Edit: I meant VK_KHR_deferred_host_operations not VK_KHR_deferred_operations

You'd need to implement KHR_buffer_device_address first, at least the non-shader side of it.

[devshgraphicsprogramming]

Since buffer device address now available in Metal 3, VK_KHR_buffer_device_address has been included in the latest version of MoltenVK. Now you should actually be able to implement VK_KHR_ray_tracing? Is there any information in this direction?

I have plans to work on this over the next six months. (Full disclosure: I'm being paid to do this.)

Seeing as it’s been nearly a year since this comment was made, it might be worth hunting for another developer to work on this.

I've been offering to implement since 2021, but seems there's no funding available.

[AntarticCoder]

@devshgraphicsprogramming

I believe VK_KHR_buffer_address has been implemented already according to MoltenVK/Layers/MVKExtensions.def on the line

MVK_EXTENSION(KHR_buffer_device_address, KHR_BUFFER_DEVICE_ADDRESS, DEVICE, 13.0, 16.0)

[natevm]

@AntarticCoder is VK_KHR_deferred_host_operations actually required for VK_KHR_acceleration_structure?

I noticed that if I remove VK_KHR_deferred_host_operations, that my vulkan ray tracing application still works. I'm only using the device-side acceleration structure builds though, ie vkCmdBuildAccelerationStructuresKHR rather than vkBuildAccelerationStructuresKHR. But that's probably the most common usage.

vkBuildAccelerationStructuresKHR does consume a VkDeferredOperationKHR for the host-side accel build, but perhaps so long as no host commands are actually deferred, then perhaps this extension isn't required for VK_KHR_acceleration_structure?

Edit: ah, nvm, I get validation errors if I remove that extension.

[AntarticCoder]

@natevm You seem to be right, but would that not be going off the specification? Or are we allowed to deviate from the specification like this?

Some applications may use deferred operations as well as that, especially if they're building on the host, so that would also break some compatabillity, I believe.

[natevm]

@AntarticCoder if it violates spec, then it's likely required to be supported.

The extension seems simple enough itself. Would require defining an opaque VkDeferredOperationKHR handle which I believe is allocated by a vkCreateDeferredOperationKHR. Then, if this type is passed to a deferrable command (eg vkBuildAccelerationStructuresKHR) then that command might return VK_OPERATION_DEFERRED_KHR rather than the normal VK_SUCCESS or etc.

From there, the command can be executed using vkDeferredOperationJoinKHR. There's a vkGetDeferredOperationResultKHR, a vkDestroyDeferredOperationKHR, and then finally a vkGetDeferredOperationMaxConcurrencyKHR which could iiuc could just return the number of available CPU cores perhaps? Seems to depend on the operation passed in to be deferred...

Might be worth making a separate issue on this, then indicating that this issue depends on that one.

[AntarticCoder]

Sure, I'll create an issue.

[rcaridade145]

A related issue from spirv-cross KhronosGroup/SPIRV-Cross#2115

[Try]

A related issue from spirv-cross KhronosGroup/SPIRV-Cross#2115

One more issue to note: spirv-cross also have no support for cullMask, for MSL backed. Also I'm not aware of any good way to implement cullMask in MSL ray-query workflow.

[devshgraphicsprogramming]

A related issue from spirv-cross KhronosGroup/SPIRV-Cross#2115

One more issue to note: spirv-cross also have no support for cullMask, for MSL backed. Also I'm not aware of any good way to implement cullMask in MSL ray-query workflow.

slap more limits into KHR_portability_subset , like that mask always must be 0xFF

[AntarticCoder]

Opened a PR: #1967

[oscarbg]

@AntarticCoder nice work.. seems next step will be implementing VK_KHR_pipeline_library?, before tackling the main raytracing extension?

[AntarticCoder]

Thanks @oscarbg. Not necessarily, only raytracing pipelines need that. Ray queries are probaly going to be implemented first due to their simpler nature.

[FunMiles]

With the M3 chips now supporting hardware accelerated Ray Tracing, this issue is becoming more pressing. Is there any current work being done on this?

[gpx1000]

We have nothing to announce just yet.