Meshlet screenspace-derived tangents

[JMS55]

• Save 16 bytes per vertex by calculating tangents in the shader at runtime, rather than storing them in the vertex data.
  • Based on https://jcgt.org/published/0009/03/04, https://www.jeremyong.com/graphics/2023/12/16/surface-gradient-bump-mapping.
• Fixed visbuffer resolve to use the updated algorithm that flips ddy correctly
• Added some more docs about meshlet material limitations, and some TODOs about transforming UV coordinates for the future.

For testing add a normal map to the bunnies with StandardMaterial like below, and then test that on both main and this PR (make sure to download the correct bunny for each). Results should be mostly identical.

normal_map_texture: Some(asset_server.load_with_settings(
    "textures/BlueNoise-Normal.png",
    |settings: &mut ImageLoaderSettings| settings.is_srgb = false,
)),

[DGriffin91]

No need to address my comments. Just wanted to throw out some ideas, possibility for the future.

[JMS55]

Did some tests visualizing the tangents directly.

Before:

After:

There's a bit of inaccuracy around the head with all the concave curves, but not too bad. Seems good to me.

[DGriffin91]

Just for clarification for my understanding, this was also the case before this PR correct?

[JMS55]

Yes. I'll need to do it as some point, low priority atm.

[JMS55]

Decided just to negate the sign of tangent.w and move on. I read a bunch of papers but idk why it's backwards.

I've confirmed that tangents pretty much match bevy_mikktspace now.

[CptPotato]

Some quick notes:

I've used tangent frames generated from pixel derivatives for a while myself, but reverted back to a full normal+tangent per vertex because of visible artifacts. The problem becomes noticeable when you have a smooth shaded mesh with a "strong" normal map. Calculating the tangent vector via derivatives means that it's not averaged for vertices and interpolated across the surface, causing visible seams between triangles (again, only if a normal map is present).

I haven't closely looked at your implementation, so it's possible that your approach is different from what I used and the issue doesn't apply here. I also think that the error is somewhat negligible - especially with the high resolution geometry that the meshlet rendering targets. So it's fine to keep as is imo., I just thought it was worth pointing out.

Save 16 bytes per vertex by calculating tangents in the shader at runtime, rather than storing them in the vertex data.

If you do go down the route of keeping the tangent frame and packing it, let me know. I think you can easily get the tangent frame down to 48 or 36 bits per vertex, possibly even smaller.

Decided just to negate the sign of tangent.w and move on. I read a bunch of papers but idk why it's backwards.

I've confirmed that tangents pretty much match bevy_mikktspace now.

There's two neat glTF test models you can use to check if your tangents are correct, NormalTangentTest & NormalTangentMirrorTest. They do have some weird seams you can ignore, only the spherical portion is important.

[JMS55]

Yeah, I'm not personally too worried about the inaccuracy. It's "good enough" for the intended use case. Nanite does have an option for explicit tangents, it wouldn't be too hard to add it back later. Not a priority right now though.

[atlv24]

this is really cool. well done

[DGriffin91]

Shadows don't seem to match current bevy main (0ebd7fc).

Current main

This PR

[JMS55]

Bevy main is using an outdated bunny model, I had forgot to regenerate the meshlet mesh before merging the last meshlet PR. The model on main is based on one of the earlier commits of the last meshlet PR, and not the last commit when it was ready to merge. I'm fairly certain that if you regenerate the meshlet mesh on main, you'd see the same issue.

As for why it looks broken, the LOD system thinks that the error from a lower LOD is not visible to the shadow map's camera, but when the shadow map is used by the main view it then has too much error. Using higher resolution shadow maps would fix this, but is more expensive. I'll probably look into adding a LOD bias for shadow views in a future PR.

[JMS55]

Bunnies on main (regenerated to ensure it's using the exact code from main)

[JMS55]

Bunnies in this PR (also freshly regenerated)

[IceSentry]

LGTM