Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Vulkan AS rebuild-on-replay: Serialise and Replay #3451

Merged
merged 5 commits into from
Oct 21, 2024

Conversation

cmannett85-arm
Copy link
Collaborator

@cmannett85-arm cmannett85-arm commented Oct 4, 2024

The VkAccelerationStructureInfo structure is serialised followed by the unified geometry readback buffer. On replay the upload buffer is populated from the capture and then copied to GPU local mem, at which point the upload mem is freed as replay cannot modify the AS input data.

When the initial state Apply() is called the ASes are built from the input data one at a time so that a single scratch buffer can be built (enlarging when needed) and re-used. Although this is slower it is necessary on Mali as it has poor space efficiency for the scratch buffer and so can easily OOM a device if all the ASes are built in a single command buffer submission.

Tested on:

  • Android Mali G715 r49p1 with various Arm tech demos (some based on UE5) and Vulkan Samples ray_queries
  • Ubuntu 24.04 NV 3090FE 535.183.06, same above but ported to x86_64

renderdoc/driver/vulkan/vk_manager.h Outdated Show resolved Hide resolved
renderdoc/driver/vulkan/vk_acceleration_structure.cpp Outdated Show resolved Hide resolved
renderdoc/driver/vulkan/vk_acceleration_structure.cpp Outdated Show resolved Hide resolved
renderdoc/driver/vulkan/vk_acceleration_structure.cpp Outdated Show resolved Hide resolved
renderdoc/driver/vulkan/vk_acceleration_structure.cpp Outdated Show resolved Hide resolved
renderdoc/driver/vulkan/vk_acceleration_structure.cpp Outdated Show resolved Hide resolved
renderdoc/driver/vulkan/vk_acceleration_structure.cpp Outdated Show resolved Hide resolved
renderdoc/driver/vulkan/vk_acceleration_structure.cpp Outdated Show resolved Hide resolved
The VkAccelerationStructureInfo structure is serialised followed by the unified geometry readback buffer.

When the initial state Apply() is called the ASes are built from the input data one at a time so that a single scratch buffer can be built (enlarging when needed) and re-used.  Although this is slower it is necessary on Mali as it has poor space efficiency for the scratch buffer and so can easily OOM a device if all the ASes are built in a single command buffer submission.
Rather than have a buffer per geometry, allocate a single block to carry all the input geometry.

Change-Id: Ibc6f221ba0a317e8006ccf4a252da71c0aefe7dc

drbrdt

Change-Id: Ia41dd97fec4f7a2b78fc505dacc0304bbc818789
If an initial content AS was destroyed during active capture then resetting before serialisation meant that VkInitialContents::Free() was called, deref-ing the AS info and causing it to be deleted.

So VkInitialContents is now reset after serialisation to ensure its lifetime.

Change-Id: I6a296bd96dd5b639f68347073298e5545e8f980d
@cmannett85-arm
Copy link
Collaborator Author

I've rebased to pick up the mutable VkInitialContents change you made, and added the single input data mem block and build-on-apply-once change as separate commits.

Then clone to captured AS.  This is much faster than building on each Apply().
renderdoc/driver/vulkan/vk_core.cpp Outdated Show resolved Hide resolved
renderdoc/driver/vulkan/vk_acceleration_structure.cpp Outdated Show resolved Hide resolved
renderdoc/driver/vulkan/vk_acceleration_structure.cpp Outdated Show resolved Hide resolved
renderdoc/driver/vulkan/vk_acceleration_structure.cpp Outdated Show resolved Hide resolved
Also:
* Use GPUBuffer instead of own radback memory mechanism, but mark as internal
* Use GPU local buffer for AS builds instead of upload memory to improve performance
@baldurk baldurk merged commit 8e5ab5d into baldurk:v1.x Oct 21, 2024
16 checks passed
@cmannett85-arm cmannett85-arm deleted the replay branch October 21, 2024 13:32
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants