EXT_texture_avif

[leon]

This extension adds support for .avif textures.

I was really hoping for jpeg xl to win the race, but google pulled it from chrome.
so the next best image format that is getting good support in browsers
https://caniuse.com/avif
is AVIF

This extension is a copy of the webp extension, but I've rewritten parts of the readme, and created a simple sample with and without png fallback.

I don't know if there are any special considerations we need to take regarding avif.

let me know if you would like me to rework anything.

[leon]

I have done a PR to three to get this added there.
mrdoob/three.js#25173

[leon]

I have also done a PR for Babylon
BabylonJS/Babylon.js#13370

[lexaknyazev]

The content of this section should not have been copy-pasted.

[leon]

I guess we are the authors?
I added myself, Don and you :)

[lexaknyazev]

Replace the copy-pasted WebP rationale with something specific to AVIF. The section should mention concrete glTF use-cases that benefit from using this extension.

[leon]

I changed it a bit, but copy is now my strong suite, feel free to update it to what you think is better.

For me the main advantage while developing mobile open gl is transfer size on 4g.
beeing able to only have to download 2mb instead of 20 is a big win.

[lexaknyazev]

That's only a win on network transmission / disk storage. AVIF decoding complexity is higher than with other formats and GPU memory usage is the same as JPEG / PNG / WebP. For example (don't copy this literally), a 3D application that features real photos / paintings may prioritize transmission size over other other metrics.

In a case of glTF, images are used as textures, for which GPU throughput is usually more important, so our general recommendation is to use KHR_texture_basisu.

This section should provide glTF use-cases that prioritize transmission / storage size and are not bound by GPU memory or throughput.

[lexaknyazev]

Given the flexibility of the format (auxiliary images, sequences, etc.), the extension spec must define an appropriate scope that is compatible with glTF texture usage.

[donmccurdy]

@lexaknyazev is it necessary to specify limitations on color space and bit depth? Or can those be inherited from the core specification, and overridden explicitly by a future extension should we choose to allow, for example, Display P3 or Rec 2100 HLG?

[lexaknyazev]

The core specification already defines color management. Other extra features that could be present in AVIF files should be addressed here.

[leon]

I added a part about this only being a replacement for jpeg / png.

I added support for the extension in gltf-pipeline today
https://github.com/leon/gltf-pipeline/tree/avif

an .avif file that is an image starts with ftypavif
whilst image sequences has another byte sequence,
I only added support for the image version for now.

[donmccurdy]

Experimental implementation in glTF Transform, for testing purposes:

# install
npm install --global @gltf-transform/cli@avif

# compress textures
gltf-transform avif input.glb output.glb

Related:

• Add EXT_texture_avif donmccurdy/glTF-Transform#771
• Replace @squoosh/lib with sharp donmccurdy/glTF-Transform#752

[donmccurdy]

As FYI: three.js is planning to add support for EXT_texture_avif in our next release, r150.

[zeux]

Out of curiosity, what is the motivation for including both WebP and AVIF as glTF extensions? Is there specific guidance on when an application should use WebP vs AVIF, and how they compare to BasisU?

[donmccurdy]

I'll copy my thoughts from this thread, they're probably more relevant here anyway —

I expect that any sufficiently large graphics application — particularly at the scale and complexity of a game — is best served by GPU texture formats like KTX2. And often in much smaller applications too, for exactly the reasons you mention. In the kind of "virtual world software" described by the OP that may well be the best choice.

However, the web has a wide range of other content patterns. Consider VFX on a fancy brand's landing page, or 3D graphic illustrations on a news article. Visitors quickly leave a site if it doesn't load almost instantly, time to first-contentful-paint should be <2s, and page size budgets are commonly measured in kilobytes, not megabytes. Under those constraints I think web developers often want the smallest transmission size they can get, and would expect to use relatively little of their VRAM budget.

Other obstacles include the comparatively easy configuration of good lossless or near-lossless compression in WebP and AVIF formats, compared to more careful tuning often required in KTX2. Raising awareness around the VRAM differences in these formats is an important and ongoing challenge, too.

I would like to find better comparisons on WebP vs. AVIF for graphics purposes, but my order of preference would be:

1. Use KTX2 when VRAM or texture GPU upload time are critical
2. Use AVIF when bundle size is critical and VRAM is not, if device supports AVIF
3. Use WebP when bundle size is critical and VRAM is not, if device doesn't support AVIF
4. Use PNG and JPEG for portability between applications

It isn't quite cut-and-dried: there are certainly cases where WebP outperforms AVIF. But I think that is the general shape of things.

[zeux]

It isn't quite cut-and-dried: there are certainly cases where WebP outperforms AVIF. But I think that is the general shape of things.

It feels a little odd to have two independent formats, AVIF and WebP, that seem to perform a similar function. This bloats the spec and results in increased complexity in the entire ecosystem, since now multiple formats of each type need to be supported. It makes sense to support one of AVIF or WebP but it feels odd to support both? Maybe the ship has sailed because WebP is already supported, and AVIF seems to compress a little better.

[bghgary]

Maybe the ship has sailed because WebP is already supported

WebP is not ratified though.

[donmccurdy]

My feeling is that the KHR_ prefix should represent extensions that Khronos believes are net-positive for the ecosystem, with a healthy respect for the increased complexity of supporting multiple similar alternatives. EXT_ extensions should represent only that "at least two vendors have chosen to implement this thing", without a value judgment from Khronos. Certainly I would not expect the entire ecosystem to implement all EXT_ extensions. In three.js we're trying to support all KHR_ extensions, but we've intentionally decided to implement some EXT_ extensions and not others, on a case by case basis.

AVIF appears to generally outperform WebP, PNG, and JPEG. Compared to KTX2 there are pros and cons. For three.js I would like to offer all of those options. We can rename this to a THREE_ prefix if there are no other interested vendors.

[lexaknyazev]

There's nothing specific to three.js here and It's reasonable to expect more vendors to support this extension over time, so let's keep it as EXT.

Use AVIF when bundle size is critical and VRAM is not

The extension spec should provide specific examples.

[aaronfranke]

@zeux I am interested in implementing EXT_texture_webp in Godot Engine, but I can't do the same with AVIF yet because Godot Engine does not support AVIF. It will take time before AVIF is a commonly enough adopted format, in the meantime WebP has been around for a decade, is more widely supported, and is still superior to PNG and JPG.

I don't see the harm in having both available, if only WebP then we're holding back progress for devices that support AVIF (so that they can't reach their full potential), and if only AVIF then we are holding back progress for devices that don't yet support AVIF (so that they are still stuck using PNG or JPG).

[donmccurdy]

I was really hoping for jpeg xl to win the race, but google pulled it from chrome.

Since it's mentioned above – some signs of life for JPEG XL:

https://webkit.org/blog/14205/news-from-wwdc23-webkit-features-in-safari-17-beta/#images

[donmccurdy]

Use AVIF when bundle size is critical and VRAM is not

The extension spec should provide specific examples.

Proposal:

• Website landing pages displaying a 3D "hero" illustration
• Authoring contexts in which lossless compression is required
• Small "islands" of 3D content within a traditional site

There's also a practical angle here — users may not have the time or ability to tune compression settings and UV layouts for optimized KTX2 compression. Platforms hosting user-generated content may have difficulty generalizing KTX2 settings without a human in the loop. But I don't think those are concerns the spec needs to call out.

I think it would also be fair to provide counter-examples. Content intended for immersive AR/VR applications should almost certainly use KTX2. The same is likely true for content intended for use in external 3D scenes where VRAM usage may be higher (or unknown).