Skip to content

Commit

Permalink
Merge pull request #1247 from madeline-underwood/raytracing_vulkan
Browse files Browse the repository at this point in the history
Raytracing vulkan_KB to review
  • Loading branch information
pareenaverma committed Sep 13, 2024
2 parents 1c0c946 + e155afc commit e37f646
Show file tree
Hide file tree
Showing 11 changed files with 144 additions and 186 deletions.
Original file line number Diff line number Diff line change
@@ -1,17 +1,17 @@
---
title: Introduction to Ray tracing with Vulkan on Android
title: Learn about Ray Tracing with Vulkan on Android

minutes_to_complete: 120

who_is_this_for: Vulkan developers who are familiar with rendering and are interested in adding ray tracing to their applications.
who_is_this_for: This Learning Path is for Vulkan developers who are familiar with rendering and are interested in deploying ray tracing in their applications.

learning_objectives:
- Learn how the Vulkan ray tracing API works.
- Learn how to use ray tracing to implement realistic shadows, reflections and refractions.
- Describe how the Vulkan ray tracing API works.
- Describe how to use ray tracing to implement realistic shadows, reflections, and refractions.
- Implement basic ray tracing effects in a Vulkan renderer.

prerequisites:
- An appropriate Android device (e.g., Vivo X100) supporting the required Vulkan extensions.
- An appropriate Android device that supports the required Vulkan extensions (for example, Vivo X100).
- Knowledge of the Vulkan API.
- A Vulkan renderer. Most code is generic and should be easy to incorporate into any deferred PBR renderer.

Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -2,28 +2,17 @@
review:
- questions:
question: >
Which is the recommended way to do ray traversal on Arm GPUs?
Which of these is the recommended way to do ray traversal on Arm GPUs?
answers:
- Ray Query
- Ray Tracing Pipeline
- Ray query
- Ray tracing pipeline
correct_answer: 1
explanation: >
Ray query is the most efficient way to implement ray traversal on Arm GPUs.
- questions:
question: >
Select the correct statement:
answers:
- A TLAS contains the model geometry
- A BLAS uses instances to group other TLASes
- A BLAS contains the model geometry
correct_answer: 3
explanation: >
BLASes (Bottom-Level Acceleration Structures) contain the actual geometry data, usually as triangles. TLASes (Top-Level Accelerations Structures) contain other BLASes and use instances to group and link them with other properties.
- questions:
question: >
How should we design the acceleration structure?
When designing acceleration structure, which of the following statements is true?
answers:
- Empty space does not matter, reduce BLAS overlap.
- Minimize empty space and minimize overlap.
Expand All @@ -32,53 +21,18 @@ review:
correct_answer: 2
explanation: >
On ray tracing, the quality of your acceleration structure can have a huge performance impact. Try to reduce overlap across BLASes and reduce empty space inside a BLAS as much as possible.
- questions:
question: >
Is bindless necessary for ray tracing?
answers:
- We do not need it for shadows, but it is needed for reflections and refractions.
- Technically no, but we need it to implement our effects.
- It is not needed, but it makes implementing our effects a lot easier.
correct_answer: 3
explanation: >
Bindless or descriptor indexing is independent of ray tracing. It is possible to implement our ray tracing effects without using it, however it will make it very easy and simple to access the data of the intercepted objects. This helps a lot when implementing reflections and refractions.
- questions:
question: >
Can reflections handle objects outside the screen?
answers:
- Ray tracing reflections can reflect objects not on the screen but Screen Space Reflections can only reflect objects on the screen.
- Both Screen Space Reflections and ray tracing reflections can reflect objects not on the screen.
- Neither Screen Space Reflections or ray tracing reflections can reflect objects on the screen.
- Neither Screen Space Reflections nor ray tracing reflections can reflect objects on the screen.
- Ray tracing reflections can only reflect objects on the screen but Screen Space Reflections can reflect objects not on the screen.
correct_answer: 1
explanation: >
Screen Space Reflections obtains the information from the G-buffer so it can only reflect object currently on the screen. Ray tracing reflections offer better quality since they can handle any object in the acceleration structure, including objects not on the screen.
- questions:
question: >
Which sentence is true for our ray tracing effects?
answers:
- In ray tracing shadows and reflections we use bindless to retrieve the material of the intercepted object and illuminate it.
- In ray tracing shadows we do not care about which objects we hit, only whether we hit an object or not.
- In ray tracing reflections we can use the flag gl_RayFlagsTerminateOnFirstHitEXT.
correct_answer: 2
explanation: >
In ray tracing reflections we need to know which object we are hitting to retrieve its material and illuminate it. In ray tracing shadows we do not care which exact object we hit, just whether we hit an object at all. This allows us to enable the gl_RayFlagsTerminateOnFirstHitEXT optimization for shadows but not for reflections.
- questions:
question: >
What is the difference between transparency and refraction?
answers:
- Light goes through a transparent object in a straight line, but refractions bend the light.
- None, refractions are how transparency is implemented in ray tracing.
- Refractions are a faster way to implement transparency.
correct_answer: 1
explanation: >
In a transparent material, light goes through in a straight line, so light rays enter and exit the material in the same direction. Refractions bend the light inside the object, so the ray exits the object in a different direction. We can use refractions to simulate ray tracing transparency, however there are simpler, more efficient ways.
# ================================================================================
# FIXED, DO NOT MODIFY
# ================================================================================
Expand Down
Original file line number Diff line number Diff line change
@@ -1,34 +1,32 @@
---
title: "Introduction: What is ray tracing?"
title: "What is ray tracing?"
weight: 2

### FIXED, DO NOT MODIFY
layout: learningpathall
---

## Introduction: What is ray tracing?

Ray tracing is a technique to render images using ray casting methods to simulate light transport. Traditionally, this is not suitable for real time or games, and is mostly used by CGI or movies.
Ray tracing is a technique to render images using ray casting methods to simulate light transport. Historically, this was not suitable for real-time computer graphics or gaming, and was mostly used by CGI in movies.

Vulkan includes a new set of extensions for ray intersection tests, designed to facilitate ray tracing.

The API allows applications to define a ray, setting an origin and a direction and then test this ray against the scene geometry. By default, the API will return the closest hit that the ray will intercept in our scene.
The API allows applications to define a ray; setting an origin and a direction, and then testing this ray against the scene geometry. By default, the API returns the closest hit that the ray intercepts in the scene.

Vulkan's ray tracing API is very flexible and allows creative uses like collision detection in physics simulations. However, the main use case for ray tracing is rendering. This is because ray tracing allows applications to simulate the behavior of light, in a way resembling physical reality.
Vulkan's ray tracing API is very flexible and enables creative use, such as collision detection in physics simulations. The the main use case for ray tracing, however, is *rendering*. This is because ray tracing allows applications to simulate the behavior of light in a way that resembles physical reality.

In the real world a light source emits light rays in all directions, these rays interact with multiple objects. When a ray intersects an object, the light will interact with it, causing the object to absorb and reflect certain amounts of light.
In the real world, a light source emits light rays in all directions. These rays interact with multiple objects. When a ray intersects an object, the object absorbs and reflects certain amounts of light.

Traditionally, developers render games using rasterization. Rasterization works by transforming the geometry into triangle primitives and projecting them onto the screen. GPUs then use a depth buffer to resolve the visibility and decide which pixels are covered by each triangle.
Traditionally, developers render games using *rasterization*. Rasterization works by transforming the geometry into triangle primitives and projecting them onto the screen. GPUs then use a depth buffer to resolve the visibility and decide which pixels are covered by each triangle.

With ray tracing, we can instead use path tracing. A path tracer does not need to use rasterization, instead it can launch rays from the camera. These rays will bounce around the scene until they produce a final image, resolving visibility using the closest hit. In the real world, rays travel from a light until they reach the camera, but this is extremely inefficient since most rays will not reach our eyes, this is why in rendering we launch rays in the reverse order, starting from the camera.
With ray tracing, you can instead use *path tracing*. A path tracer does not need to use rasterization, instead it can launch rays from the camera. These rays bounce around the scene until they produce a final image, resolving visibility using the closest hit. In the real world, rays travel from a light until they reach the camera, but this is extremely inefficient as most rays do not reach our eyes. This is why in rendering you can launch rays in the reverse order, starting from the camera.

Path tracing is extremely costly since we will need to launch thousands of rays per pixel to produce a non-noisy image. Rendering a frame in real time using path tracing is not feasible even on desktop high end GPUs. The common approach is to have a hybrid renderer, with a traditional rasterization pass to resolve visibility and compute the G-buffer, and then implement each ray tracing effect as a separate post-process.
Path tracing is extremely costly as it requires thousands of rays per pixel to be launched to produce a non-noisy image. Rendering a frame in real time using path tracing is just not feasible, even on desktop high-end GPUs. The common solution therefore is to have a hybrid renderer, with a traditional rasterization pass to resolve visibility and compute the G-buffer, and then implement each ray tracing effect as a separate post-process.

{{< tabpane >}}
{{< tab header="Example 1: RT ON" title="Example 1: ray tracing ON" img_src="/learning-paths/smartphones-and-mobile/ray_tracing/images/city_rt_on.png">}}{{< /tab >}}
{{< tab header="Example 1: RT OFF" title="Example 1: ray tracing OFF" img_src="/learning-paths/smartphones-and-mobile/ray_tracing/images/city_rt_off.png">}}{{< /tab >}}
{{< tab header="Example 2: RT ON" title="Example 2: ray tracing ON" img_src="/learning-paths/smartphones-and-mobile/ray_tracing/images/bonza_rt_on.png">}}{{< /tab >}}
{{< tab header="Example 2: RT OFF" title="Example 2: ray tracing OFF" img_src="/learning-paths/smartphones-and-mobile/ray_tracing/images/bonza_rt_off.png">}}{{< /tab >}}
{{< tab header="Example 3: RT ON" title="Example 3: ray tracing ON" img_src="/learning-paths/smartphones-and-mobile/ray_tracing/images/immortalis_rt_on.png">}}{{< /tab >}}
{{< tab header="Example 3: RT OFF" title="Example 3: ray tracing OFF" img_src="/learning-paths/smartphones-and-mobile/ray_tracing/images/immortalis_rt_off.png">}}{{< /tab >}}
{{< tab header="Example 1: RT ON" title="Example 1: Ray tracing ON" img_src="/learning-paths/smartphones-and-mobile/ray_tracing/images/city_rt_on.png">}}{{< /tab >}}
{{< tab header="Example 1: RT OFF" title="Example 1: Ray tracing OFF" img_src="/learning-paths/smartphones-and-mobile/ray_tracing/images/city_rt_off.png">}}{{< /tab >}}
{{< tab header="Example 2: RT ON" title="Example 2: Ray tracing ON" img_src="/learning-paths/smartphones-and-mobile/ray_tracing/images/bonza_rt_on.png">}}{{< /tab >}}
{{< tab header="Example 2: RT OFF" title="Example 2: Ray tracing OFF" img_src="/learning-paths/smartphones-and-mobile/ray_tracing/images/bonza_rt_off.png">}}{{< /tab >}}
{{< tab header="Example 3: RT ON" title="Example 3: Ray tracing ON" img_src="/learning-paths/smartphones-and-mobile/ray_tracing/images/immortalis_rt_on.png">}}{{< /tab >}}
{{< tab header="Example 3: RT OFF" title="Example 3: Ray tracing OFF" img_src="/learning-paths/smartphones-and-mobile/ray_tracing/images/immortalis_rt_off.png">}}{{< /tab >}}
{{< /tabpane >}}
Original file line number Diff line number Diff line change
@@ -1,5 +1,5 @@
---
title: "Setup: Enabling ray tracing"
title: "Setup: enabling ray tracing"
weight: 3

### FIXED, DO NOT MODIFY
Expand All @@ -8,13 +8,13 @@ layout: learningpathall

## Setup: Enabling ray tracing

Vulkan uses the same API for ray tracing on PC and mobile devices. This makes it extremely easy to implement and test our ray tracing effects on PC and deploy them on mobile. Porting existing ray tracing effects from PC to mobile should also be simple.
Vulkan uses the same API for ray tracing on PC and mobile devices. This makes it extremely easy to implement and test ray tracing effects on PC and deploy them on mobile. Porting existing ray tracing effects from PC to mobile should also be simple.

Arm® Mali™-G7-series GPUs after the [Arm® Mali™-G715](https://developer.arm.com/Processors/Mali-G715) might or might not support for ray tracing, depending on the phone. Immortalis GPUs like [Arm® Immortalis-G715](https://developer.arm.com/Processors/Immortalis-G715), [Arm® Immortalis-G720](https://developer.arm.com/Processors/Immortalis-G720) or [Arm® Immortalis-G925](https://developer.arm.com/Processors/Immortalis-G925) always support ray tracing. These GPUs are available in multiple devices already in the market. Moreover, ray tracing is a promising modern technology, so multiple GPU vendors are supporting the API. Most recent high end Android smartphones support ray tracing, making it a feature game developers can rely on.
Arm® Mali™-G7-series GPUs after the [Arm® Mali™-G715](https://developer.arm.com/Processors/Mali-G715) might or might not support ray tracing, depending on the phone model. Immortalis GPUs such as [Arm® Immortalis-G715](https://developer.arm.com/Processors/Immortalis-G715), [Arm® Immortalis-G720](https://developer.arm.com/Processors/Immortalis-G720), or [Arm® Immortalis-G925](https://developer.arm.com/Processors/Immortalis-G925) support ray tracing. These GPUs are available in multiple devices already in the market. Moreover, ray tracing is a promising modern technology, so multiple GPU vendors support the API. Most recent high-end Android smartphones support ray tracing, making it a feature game developers can rely on.

Vulkan offers ray tracing as a series of extensions, making it easy to query for support and to enable it. The most relevant extensions are: `VK_KHR_acceleration_structure`, `VK_KHR_ray_query` and `VK_KHR_ray_tracing_pipeline`.
Vulkan offers ray tracing as a series of extensions, making it easy to query for support and to enable it. The most relevant extensions are: `VK_KHR_acceleration_structure`, `VK_KHR_ray_query`, and `VK_KHR_ray_tracing_pipeline`.

We can query the physical device to check if it supports the necessary extensions. Here is a helper function:
You can query the physical device to check if it supports the extensions. Here is a helper function:

``` cpp
uint32_t device_extension_count;
Expand All @@ -37,7 +37,7 @@ auto is_extension_supported =
};
```
We can use it to query the ray tracing extensions' features:
You can use it to query the ray tracing extensions' features:
``` cpp
VkPhysicalDeviceFeatures2 features2{VK_STRUCTURE_TYPE_PHYSICAL_DEVICE_FEATURES_2};
Expand All @@ -64,7 +64,7 @@ if (is_extension_supported(VK_KHR_RAY_TRACING_PIPELINE_EXTENSION_NAME))
vkGetPhysicalDeviceFeatures2(physical_device, &features2);
```

Finally, enable the extensions when creating the logical device:
Finally, you can enable the extensions when creating the logical device:

``` cpp
std::vector<const char *> enabled_extensions{};
Expand Down
Loading

0 comments on commit e37f646

Please sign in to comment.