StandardMaterial Light Transmission

[coreh]

Objective

This PR adds both diffuse and specular light transmission capabilities to the StandardMaterial, with support for screen space refractions. This enables realistically representing a wide range of real-world materials, such as:

• Glass; (Including frosted glass)
• Transparent and translucent plastics;
• Various liquids and gels;
• Gemstones;
• Marble;
• Wax;
• Paper;
• Leaves;
• Porcelain.

Unlike existing support for transparency, light transmission does not rely on fixed function alpha blending, and therefore works with both AlphaMode::Opaque and AlphaMode::Mask materials.

Solution

• Introduces a number of transmission related fields in the StandardMaterial;
• For specular transmission:
  • Adds logic to take a view main texture snapshot after the opaque phase; (in order to perform screen space refractions)
  • Introduces a new Transmissive3d phase to the renderer, to which all meshes with transmission > 0.0 materials are sent.
  • Calculates a light exit point (of the approximate mesh volume) using ior and thickness properties
  • Samples the snapshot texture with an adaptive number of taps across a roughness-controlled radius enabling “blurry” refractions
• For diffuse transmission:
  • Approximates transmitted diffuse light by using a second, flipped + displaced, diffuse-only Lambertian lobe for each light source.

To Do

• Figure out where fresnel_mix() is taking place, if at all, and where dielectric_specular is being calculated, if at all, and update them to use the ior value (Not a blocker, just a nice-to-have for more correct BSDF)
  • To the _best of my knowledge, this is now taking place, after 964340c. The fresnel mix is actually "split" into two parts in our implementation, one (1 - fresnel(...)) in the transmission, and fresnel() in the light implementations. A surface with more reflectance now will produce slightly dimmer transmission towards the grazing angle, as more of the light gets reflected.
• Add transmission_texture
• Add diffuse_transmission_texture
• Add thickness_texture
• Add attenuation_distance and attenuation_color
• Connect values to glTF loader
  • transmission and transmission_texture
  • thickness and thickness_texture
  • ior
  • diffuse_transmission and diffuse_transmission_texture (needs upstream support in gltf crate, not a blocker)
• Add support for multiple screen space refraction “steps”
• Conditionally create no transmission snapshot texture at all if steps == 0
• Conditionally enable/disable screen space refraction transmission snapshots
• Read from depth pre-pass to prevent refracting pixels in front of the light exit point
• Use interleaved_gradient_noise() function for sampling blur in a way that benefits from TAA
• Drill down a TAA #define, tweak some aspects of the effect conditionally based on it
• Remove const array that's crashing under HLSL (unless a new naga release with [wgsl-in] Use deterministic ordering for dependency ordering. gfx-rs/naga#2496 comes out before we merge this)
• Look into alternatives to the switch hack for dynamically indexing the const array (might not be needed, compilers seem to be decent at expanding it)
• Add pipeline keys for gating transmission (do we really want/need this?)
• Tweak some material field/function names?

A Note on Texture Packing

This was originally added as a comment to the specular_transmission_texture, thickness_texture and diffuse_transmission_texture documentation, I removed it since it was more confusing than helpful, and will likely be made redundant/will need to be updated once we have a better infrastructure for preprocessing assets

Due to how channels are mapped, you can more efficiently use a single shared texture image
for configuring the following:

• R - specular_transmission_texture
• G - thickness_texture
• B - unused
• A - diffuse_transmission_texture

The KHR_materials_diffuse_transmission glTF extension also defines a diffuseTransmissionColorTexture,
that we don't currently support. One might choose to pack the intensity and color textures together,
using RGB for the color and A for the intensity, in which case this packing advice doesn't really apply.

---

Changelog

• Added a new Transmissive3d render phase for rendering specular transmissive materials with screen space refractions
• Added rendering support for transmitted environment map light on the StandardMaterial as a fallback for screen space refractions
• Added diffuse_transmission, specular_transmission, thickness, ior, attenuation_distance and attenuation_color to the StandardMaterial
• Added diffuse_transmission_texture, specular_transmission_texture, thickness_texture to the StandardMaterial, gated behind a new pbr_transmission_textures cargo feature (off by default, for maximum hardware compatibility)
• Added Camera3d::screen_space_specular_transmission_steps for controlling the number of “layers of transparency” rendered for transmissive objects
• Added a TransmittedShadowReceiver component for enabling shadows in (diffusely) transmitted light. (disabled by default, as it requires carefully setting up the thickness to avoid self-shadow artifacts)
• Added support for the KHR_materials_transmission, KHR_materials_ior and KHR_materials_volume glTF extensions
• Renamed items related to temporal jitter for greater consistency

Migration Guide

• SsaoPipelineKey::temporal_noise has been renamed to SsaoPipelineKey::temporal_jitter
• The TAA shader def (controlled by the presence of the TemporalAntiAliasSettings component in the camera) has been replaced with the TEMPORAL_JITTER shader def (controlled by the presence of the TemporalJitter component in the camera)
• MeshPipelineKey::TAA has been replaced by MeshPipelineKey::TEMPORAL_JITTER
• The TEMPORAL_NOISE shader def has been consolidated with TEMPORAL_JITTER

[JMS55]

A couple of notes:

• We should rename things to show that this is specular transmission only, not diffuse.
• For the dithering, look into the PCF code. Using the same stochastic IGN sampling when TAA is enabled might lead to better dithering.
• Look into downsampled/blurred mip pyramids instead of dithering. Might be lower artifacts / cheaper (if lots of transmissive materials).
• Look into how to handle total internal reflection.
• This falls under the umbrella of screen space global illumination (SSGI). Along with SSAO (occlusion), SSIL (diffuse), and SSR (specular). SSR techniques in particular are very applicable to you, see the links below. For now whatever you get working is great, but eventually I'd like to combine them all into one module/component, and try to share code where possible. Something to refactor when all that is implemented, however :)
  • https://www.ea.com/frostbite/news/stochastic-screen-space-reflections
  • https://github.com/GPUOpen-Effects/FidelityFX-SSSR/blob/master/docs/FFX_SSSR_Technology.pdf
• Another way of faking refraction, is to use the environment map instead of sampling from the screen. I'm not sure this is worth bothering to implement though, as it's lower quality then a screen space method.

[zArubaru]

Great work @coreh and everyone else involved ❤️🎉