Impact of colorspace on WebCodecs

[tidoust]

The WebCodecs proposal is actively being developed in the WICG. It defines an API for encoding and decoding audio and video. The HTML draft is outdated for now, the latest version of the API is actually being discussed in a Google doc.

The API exposes decoded video frames. The // TODO: color space metadata comment suggests that the color space will be exposed somehow.

[palemieux]

The WebCodecs CG report is at https://wicg.github.io/web-codecs/. The Pixel Format section only references Planar 4:2:0 YUV.

[palemieux]

@chcunningham Any input on how WebCodecs plans to handle WCG/HDR?

[chcunningham]

In chromium, under the hood the decoders already support these formats (same pipeline from <video>). But we're missing a number of things in the WebCodecs spec. At a glance:

• I think we'll need to include something like this VideoColorSpace structure found in chromium code. This would then be set on the VideoDecoderConfig and VideoFrame objects and we also need to update rendering code paths to use that for color-correct rendering.
• We'll also need to add additional pixel formats.

Timing-wise, SDR is our current focus.

[jyrkialakuijala]

Absolute color XYB colorspace (used internally in JPEG XL) would allow SDR and HDR use the same colorspace, possibly making it more straightforward to compose graphics from SDR and HDR sources and other graphics primitives.

[chrisn]

There's now a proposal for HDR support in WebCodecs - see w3c/webcodecs#384.