HDR (DCI-P3 & Rec. 2020) support

[WesselKroos]

Waiting on the completion of the implementation of HDR support in the canvas element:
https://chromestatus.com/feature/5703719636172800
whatwg/html#9461
https://github.com/WICG/canvas-color-space/blob/main/CanvasColorSpaceProposal.md
https://github.com/w3c/ColorWeb-CG/blob/main/hdr_html_canvas_element.md

• Workaround: Separate filter settings for HDR videos.
  Which with users can finetune the minimum/maximum rgb values.
  Note: The color space conversion remains incorrect.
• Support the DCI-P3 (p3) color space
  (Currently not supported by Firefox)
  Firefox bug: https://bugzilla.mozilla.org/show_bug.cgi?id=1771373
• Support the Rec. 2020 (rec2020) color space (Currently not supported by any browser)

[WesselKroos]

There is another feature for the Display P3 colorspace support in canvas elements that was released in Chrome 104 (august 2022):
https://chromestatus.com/feature/4814886323355648

Since that is easier to implement it has been added to the 2.37.25 version

[Askejm]

Any estimate of when this will be added? Have heard a lot of people using this to help uneven burn-in on ultrawide OLEDs but then we can't get proper HDR

[WesselKroos]

Regarding Firefox, that's up to the developers at Mozilla. I think you can follow this Firefox bug:
https://bugzilla.mozilla.org/show_bug.cgi?id=1771373

Once they've implemented display-p3 support for the canvas it should work immediately.

[WesselKroos]

Then, are you using NVidia RTX to watch a non-hdr video in hdr?

[Askejm]

I am on a 3090, but I have RTX super resolution and HDR disabled

[WesselKroos]

Interesting, I guess Firefox uses MPO's to display HDR videos directly through the graphics card driver so that Firefox doesn't have to do any color conversion. But do the colors in the canvas match with the video colors?
If not you could check if setting Firefox's webgl.colorspaces.prototype flag to True in about:config helps. It should promote the WebGL canvas to the display-p3 colorspace, but it seems to be in an experimental fase currently.

[Askejm]

But do the colors in the canvas match with the video colors?

Sorry, I'm not quite sure what this means? If i turn ambient lights on or off now, i see no difference in colors

[WesselKroos]

Sorry, I'm not quite sure what this means? If i turn ambient lights on or off now, i see no difference in colors

The colors of the ambient light don't always match with the colors of the video when I force HDR colors in Chrome on my non-HDR monitor. So, I'm wondering if that is also the case on HDR monitors.

For example, with chrome://flags/#force-color-profile set to ITU-R BT.2020 I get these differences:

[Askejm]

Well here I took some screenshots without webgl.colorspaces.prototype:


However these are SDR screenshots. I had to use OBS to turn them into SDR. I'm not sure which tone mapping algorithm it uses, but it is clear that ambient lights has less punchy highlights since it appears to be SDR. It's kinda hard for me to see the difference, even in your screenshot, but it appears that there is mostly a luminance difference. Now at this point i realized its probably better to take a picture of my screen to better show how the colors are represented in person. I forgot to take a screenshot with the flag set to true
False:

True:

They appear pretty much identical to me. I took these with my camera, where all the settings where kept the same except the third picture where i bumped up the iso to get more of the blacks

[WesselKroos]

It's indeed hard to see in the screenshots and photographs. You can see the difference in my screenshot at the left side of the video. It's mainly visible in the purples/pinks and the greens, which are grayish/duller in the ambient light than in the video when the Rec.2020 colorspace simulation is enabled in Chrome:

That's because the saturated colors of the Rec.2020 color space in the video are getting limited to the Rec.709 (srgb) colorspace in the ambient light canvas. Here is the difference in a graph:

I can also see a similar effect in your screenshot. The only difference is that the screenshot and ambient light canvas is limited to the Rec.709 colorspace, causing getting reds to be crushed in the ambient light canvas. Because you have a HDR monitor:

The photographs do look identical though. So I guess that the webgl.colorspaces.prototype is flag isn't doing anything at the moment.

But I think you are not watching a HDR video, because your YouTube player is not showing the HDR quality label. It could be that the HDR video quality is not available in Firefox. I see these options and label when I enable the HDR colorspace simulation in Chrome for this video: https://www.youtube.com/watch?v=1MieluM0c6c

[Askejm]

Hmmm... this is weird
I go to this LG HDR video, but it doesnt say its hdr anywhere in the quality options

However, taking a screenshot gives the blown out highlights that I get when screenshotting HDR content

But despite this, the highlights still appear a lot more toned down that downloading the video and playing it in VLC
https://github.com/WesselKroos/youtube-ambilight/assets/55248977/d6fe2da6-8b0b-4eb0-bb85-839c63902106
But if i compare it to Chrome, Chrome has even more toned down highlights
https://github.com/WesselKroos/youtube-ambilight/assets/55248977/9f4fec18-2ff4-418f-a51e-4ae42d2f71f0

[WesselKroos]

Then you are not watching a HDR YouTube video.
Maybe you need to change some settings in your LG monitor or Windows settings, like this guy: https://www.youtube.com/watch?v=ZO-x-pViufI

[WesselKroos]

Chromium issue 358587920 might cause some color differences between the video and the ambient light as well.

[mdrejhon]

• Support the Rec. 2020 (rec2020) color space (Currently not supported by any browser)

Creator of TestUFO here. Chrome/Edge is at "supported via feature flag" status now since version 110, for supporting rec2100-pq and rec2100-hlg <CANVAS> for framebuffer format, and rec2020 colorspace for colorspace format. In other words, draw calls are preserved as rec2020 on a rec2100 framebuffer.

chrome://flags/#enable-experimental-web-platform-features

And confirmed colorspace in tests:

Instructions to enable HDR + rec2100 are posted at the top of the HDR beta version of TestUFO (beta.testufo.com) as well as its new WCG/HDR tester (beta.testufo.com/hdr).

rec2100(-pq,-hlg,-linear) tends to be a superset of rec2020 with HDR extensions. Basically, rec2020 is WCG and rec2100 adds HDR while keeping rec2020 WCG. What this means (in browser contexts, at least) is rec2100 implies rec2020.

Since I have access to external laboratory equipment, I have successfully tested that 10,000-nit pixels (rec2100) with rec2020 colorspace, does format fine at the DisplayPort/HDMI cable level -- although will be severely clipped by consumer displays. No consumer display can support the whole color gamut nor brightness range of rec2100 gamuts+hdr.

• Use rec2100 canvas (create canvas)
• Use rec2020 CSS4 color values (draw operations)

And it works in Chrome/Edge/Brave.

Not all GPUs support rec2020, mind you, so the rec2100 framebuffer (using rec2020 color gamut) will cross-convert to whatever OS gamut uses. Here's an example:

On Chrome running on Mac XDR displays (e.g. MacBook Pros), the OS will cross-convert rec2020 to display-p3 HDR, so it will work on MacBooks too (HDR preserved, WCG converted). Link to photo of MacBook running Chrome doing WCG+HDR CANVAS, using rec2100 hdr, rec2020 colorspace, and displayed by OS in display-p3 (downconverted gamut).

But, technically, rec2020 (via rec2100-pq, rec2100-hlg canvas) is now finally working in Chrome/Edge/Brave.

You can draw to canvas using "color(rec2020 r g b)" anywhere you use "#rrggbb" (fillStyle etc) and even use r/g/b numbers bigger than 1.0 for colors brighter than #FFFFFF

[WesselKroos]

@mdrejhon I did some tests with the rec2100 framebuffer, but the canvas is always fainted or dark on my monitor.
I only have a Display P3 D65 color profile for my monitor (MSI Optix MAG271CQR), so maybe that invalidates these results?

colorSpace drawingBufferColorSpace unpackColorSpace	chrome://flags/ #force-color-profile	YouTube quality	Video	Canvas
srgb	sRGB	SDR	Correct	Correct
srgb	Display P3 D65	SDR	Faint	Faint
srgb	ITU-R BT.2020	SDR	Correct?	Faint
srgb	scRGB linear (HDR where available)	HDR	Correct	Slightly fainted
srgb	HDR10 (HDR where available)	HDR	Over-bright	Over-bright
-
display-p3	sRGB	SDR	Correct	Correct
display-p3	Display P3 D65	SDR	Faint	Faint
display-p3	ITU-R BT.2020	SDR	Correct?	Faint
display-p3	scRGB linear (HDR where available)	HDR	Correct	Slightly faint
display-p3	HDR10 (HDR where available)	HDR	Over-bright	Over-bright
-
rec2100-hlg / rec2100-pq	sRGB	SDR	Correct	Faint
rec2100-hlg / rec2100-pq	Display P3 D65	SDR	Faint	Dark
rec2100-hlg / rec2100-pq	ITU-R BT.2020	SDR	Correct?	Faint
rec2100-hlg / rec2100-pq	scRGB linear (HDR where available)	HDR	Correct	Dark
rec2100-hlg / rec2100-pq	HDR10 (HDR where available)	HDR	Over-bright	Dark

[mdrejhon]

Very interesting test results! The color space is limited by the monitor capabilities. HDR usually only becomes impressive to consumers on 1000+ nit displays.

The way the color space is converted varies, but did you retest with a true HDR canvas framebuffer capable of brighter than sRGB white?

Try Chrome on a MacBook with XDR display, with the Experimental Web Platform Features flag turned on in Chrome://flags to suddenly brighten the canvas when drawing HDR .avif images or using fillStyle with rec2020 colors).

It does become a conversion rec2020 canvas -> display-p3 HDR display... though you can now finally use brighter lumens and colors outside of sRGB color space when drawing on canvas. Still needs improvement but it's more correct behaviour.

[WesselKroos]

I did set the drawingbuffers to rec2100-hlg in the 2d canvas context option colorSpace and in the WebGL canvas context properties
drawingBufferColorSpace and
unpackColorSpace. (Those are the last 5 rows in the test results.)

But the canvas always remains very dark when it uses a rec2100-hlg buffer. I would expect the canvas to be overly bright since the colors should extend beyond the color space my monitor is capable of. (Just like all the other overly bright colors in the browser and webpage when I force the color profile via the flag to HDR10 (HDR where available).)

So I guess Chromium's experimental implementation is still incomplete. Maybe Chromium or the gpu driver is mapping the rec2100-hlg drawbuffer of the 2d canvas to a srgb or display-p3 buffer in the drawImage or rendering pass? It could explain why the colors are so dark, because maximum values in the rec2020 color space are lower than the maximum values of the rec2100-hlg buffer.

PS: I don't have any Apple devices or HDR10 capable devices, so I can't validate if the overly bright colors are correct. But I was able to validate that the overly bright pixels in the video are not overly bright in the canvas because all the canvas colors were pretty dark.

[Askejm]

If you need a HDR display to test on, I have an AW3423DWF which will manage 800-1000 nits (but on smaller window sizes)

[WesselKroos]

@Askejm Could you check if you are able to get YouTube's HDR quality options on a Chromium browser for the HDR video https://www.youtube.com/watch?v=1MieluM0c6c ?

If not:

• Make sure that these 2 settings in the Window settings app are displayed as Supported:
  
• Or alternatively you might need to go to the url chrome://flags/#force-color-profile , set it to HDR10 (HDR where available), reload the browser and reload the YouTube webpage.

[WesselKroos]

@Askejm If you can get YouTube's HDR quality options to work, I can create a test version (hopefully this week) in which you can switch to various settings that were till now automatically decided for HDR videos. That way you can test what works and what doesn't. These 3 advanced settings will be available under the Quality section:

Update:

• WebGL bit depths higher than 8 bits are always black on SDR video textures.
• WebGL bit depths higher than 16 bits are black on on HDR video textures.
• WebGL mipmap generation fails on HDR videos textures.
• WebGL bit depths higher than 8 bits with HDR video frames are only visible when using an intermediate canvas with a CanvasRenderingContext2D as texture source in WebGL. But on my monitor the highlights always seem to be clamped to an sRGB color profile.

Potential combinations that might work for HDR videos:

These are the only combination of settings that display overbright highlights in the canvas on my Display P3 capable display. They might be correctly visible on a HDR10 capable display.

Color profile	Color space	Color bit depth	Color conversion patch
HDR10 (HDR where available)	Display P3 D65	8 bits 10 bits 16 bits	Enabled
HDR10 (HDR where available)	Display P3 D65	8 bits	Disabled

And these combinations are fainted on my Display P3 capable display, but might display correctly on a HDR10 capable display:

Color profile	Color space	Color bit depth	Color conversion patch
HDR10 (HDR where available)	Rec. 2100 PG	8 bits 10 bits 16 bits	Enabled
HDR10 (HDR where available)	Rec. 2100 PG	8 bits	Disabled

[WesselKroos]

Some additional notes about video colors in Chromium (excluding the canvas colors) on a Display P3 D65 or ITU-R BT.2020 display:

Since Chromium version 120 the colors of a srgb video are only mapped to the full color gamut of the display while the flag chrome://flags/#use-angle is set to D3D11 and the browser displays the video via a MPO "Multi-plane overlay", also known as "direct video overlay". Because then the video frames are directly drawn by the graphics card instead of the browser.
There are several ways to disable the video MPO in Chromium: Scrolling the video half off-screen, overlaying the account menu in YouTube, resizing the browser tab, setting the angle flag to any other value than D3D11.
So all these situations cause a temporary change in video colors.

This bug was reported to me about a month ago:
"this video has exactly the same codec and also the colour changes but not so extreme, more like skin colour getting a bit more pale and then back again" - @0x2bilal
And I've verified and forwarded this bug to the Chromium developers a week later: https://issues.chromium.org/issues/358587920

[0x2bilal]

Hey there, i need to appologize that i didnt reply to the last mail, i didnt mess further with the arc browser, since I am in my exams phase of this semester xD. So thank you for updating me again and for the further information! kind greetings 🙂 Von Outlook<http://aka.ms/weboutlook> gesendet.

…

________________________________ Von: Wessel Kroos ***@***.***> Gesendet: Montag, 9. September 2024 23:26 An: WesselKroos/youtube-ambilight ***@***.***> Cc: Bilal Talleb ***@***.***>; Mention ***@***.***> Betreff: Re: [WesselKroos/youtube-ambilight] HDR (DCI-P3 & Rec. 2020) support (Issue #129) Some additional notes about video colors in Chromium (excluding the canvas colors) on a Display P3 D65 or ITU-R BT.2020 display: Since Chromium version 120 the colors of a srgb video are only mapped to the full color gamut of the display while the flag chrome://flags/#use-angle is set to D3D11 and the browser displays the video via a MPO "Multi-plane overlay", also known as "direct video overlay". Because then the video frames are directly drawn by the graphics card instead of the browser. There are several ways to disable the video MPO in Chromium: Scrolling the video half off-screen, overlaying the account menu in YouTube, resizing the browser tab, setting the angle flag to any other value than D3D11. So all these situations cause a temporary change in video colors. This bug was reported to me about a month ago: "this video has exactly the same codec and also the colour changes but not so extreme, more like skin colour getting a bit more pale and then back again"<#252 (comment)> - @0x2bilal<https://github.com/0x2bilal> And I've verified and forwarded this bug to the Chromium developers a week later: https://issues.chromium.org/issues/358587920 — Reply to this email directly, view it on GitHub<#129 (comment)>, or unsubscribe<https://github.com/notifications/unsubscribe-auth/A7ACJ2NTVXUGSDIFTPOSA6TZVYHAZAVCNFSM6AAAAABJL2K232VHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMZDGMZZGE2DKMJWHE>. You are receiving this because you were mentioned.Message ID: ***@***.***>

[WesselKroos]

@Askejm Here is a test version of the extension with the new experimental color options.
Let me know if you can find a combination of settings that display the correct colors on your HDR10 monitor, if you've got the time to test it out. The combinations I'm most optimistic about are listed in my previous comment

youtube-ambilight-v2.38.11.color-experiments-mv3.zip
To install the extension:

• Unpack the zip to a folder
• In any chromium based browser go to the url chrome://extensions/
• Turn on the Developer mode toggle
• Click on Load unpacked and select the unzipped folder

Update: After that, make sure to enable the flag at chrome://flags/#enable-experimental-web-platform-features and restart the browser.

And this might be a good test video. It clearly shows when a color is brighter or darker in the video than in the ambient light:
https://www.youtube.com/watch?v=PoMPrr-Uefs&t=114s

[WesselKroos]

@Askejm A friendly reminder, could you look at the experimental HDR color options if you get the time someday?

[Askejm]

@WesselKroos Sorry for the late response, I've been busy the last couple weeks. I'm gonna test this now. As I mentioned, I'm using the AW3423DWF. HDR Peak 1000, Console Mode and Source Tone Map are all enabled, you can see a review of its HDR performance at this level here. But the main takeaway is 1000 nit at 2% window and 459 nit at 10% window. So it can only reach these crazy brightness levels at very small window sizes.
After enabling that HDR10 flag, the HDR option appears.
I downloaded your test extension. These are the settings I'm using
sRGB and P3 look the same to me, and so does all the color bit options. I'm gonna keep it at 10-bit though.
Rec 2100 PQ and HLG both cause the extension to crash.

It seems that no matter what I do, the video is always slightly darker than ambient lights. I tried eyeballing the HDR filter to roughly match the video and got 85% brightness 115% saturation, however this was only for the bright parts of the video (like 1:15)

Earlier or later in the video, the video appears slightly brighter when the video is dark. (like 0:04)

I took some pics and now I'm getting different findings? I truly don't know at this point so I'm just gonna put them here. I just took these pics on my phone, but if you'd like to I can take a RAW image on manual with my Nikon Z5 and then color correct it to jpg to look the closest to real life. However I skipped that for this as it's quite time consuming, especially for so many pictures.

[Askejm]

I also tried checking out the angle backend

• OpenGL: Super low contrast
• D3D11: Seems to be default, although notably SDR elements are a lot darker even on HDR Peak 1000 than on Firefox for some reason.
• D3D9: Doesn't have the aforementioned effect with SDR, but has some weird colors (seems like they are lower contrast and lower saturation?)
• D3D11on12: same as D3D11

Not sure if this is for any use but I figured that I might as well

[WesselKroos]

@Askejm Regarding the Rec 2100 PQ and HLG crash, you'll need to enable the flag at chrome://flags/#enable-experimental-web-platform-features Only then you should be able to reach the Rec2020 color gamut.

Regarding the low contrasted/satured colors with other ANGLE backend options. That's correct, OpenGL and D3D9 only support the sRGB color gamut.

[Askejm]

@WesselKroos Woah! Rec2100 looks wayyyy better. Rec 2100 PQ seems to match nearly perfectly, and HLG seems to not still match perfectly but still closer.

HLG

PQ

Note: these were done in normal desktop HDR mode

[WesselKroos]

@WesselKroos Woah! Rec2100 looks wayyyy better. Rec 2100 PQ seems to match nearly perfectly, and HLG seems to not still match perfectly but still closer.

Interesting, that looks a lot better.
In PQ mode green and purple colors still looks slightly more different (desatured) than in the video.

You might get a better result with 16bit colors and/or with the Color conversion patch disabled.
Could you make pictures of the following combinations:

Color space	Color bit depth	Color conversion patch
Rec. 2100 PG	8 bits	Enabled
Rec. 2100 PG	10 bits	Enabled
Rec. 2100 PG	16 bits	Enabled
Rec. 2100 PG	8 bits	Disabled
Rec. 2100 PG	10 bits	Disabled
Rec. 2100 PG	16 bits	Disabled
Rec. 2100 HLG	8 bits	Enabled
Rec. 2100 HLG	10 bits	Enabled
Rec. 2100 HLG	16 bits	Enabled
Rec. 2100 HLG	8 bits	Disabled
Rec. 2100 HLG	10 bits	Disabled
Rec. 2100 HLG	16 bits	Disabled

[Askejm]

Sure. I'll take these with my camera then, but that's gonna have to be later

[Askejm]

@WesselKroos Here are the results.

Color Space	Color Bit Depth	Color Conversion Patch	Sample Image
Rec. 2100 PG	8 bits	Enabled
Rec. 2100 PG	10 bits	Enabled
Rec. 2100 PG	16 bits	Enabled
Rec. 2100 PG	8 bits	Disabled
Rec. 2100 PG	10 bits	Disabled	Black
Rec. 2100 PG	16 bits	Disabled	Black
Rec. 2100 HLG	8 bits	Enabled
Rec. 2100 HLG	10 bits	Enabled
Rec. 2100 HLG	16 bits	Enabled
Rec. 2100 HLG	8 bits	Disabled
Rec. 2100 HLG	10 bits	Disabled	Black
Rec. 2100 HLG	16 bits	Disabled	Black

These are taken on my Z5 shot on a tripod with fully manual settings. Color balance is 6000K. I graded it a little bit to best match real life based off the Rec. 2100 PG 10 bit enabled and exported to JPG 100% in color space Display P3. This was some very minor changes to brightness and saturation of various colors. Also disclaimer this was also graded on my AW3423DWF. And if you want the raw files for some reason I'll upload them here but probably wont keep forever

IMO the Rec. 2100 PQ 8 bits disabled is the closest to perfect. I think I can still see a little difference, especially when yellow is on the bottom. However compared to color conversion patch enabled the yellows are just completely dampened with enabled.

If I was gonna do this again it'd probably shoot slightly out of focus, because right now there is a bit of screen door effect, maybe you can blur it to get a better blend. Either way the image gets as close to real life as I can make. It's important to note that the colors blend far better in real life, most notably between blue and yellow. Hope you find this useful!

[WesselKroos]

@Askejm Wow, I don't think I'll need the raw pictures since these non-raw pictures already look perfect. Thanks for taking the time to make these for me.

The difference is clearly visible between the pictures with the Color Conversion Patch being disabled or enabled.

The difference between the PQ/HLG color gamuts are interesting when you view at the pictures with a box blur and removed black borders. Here are the edited pictures circulated with a red/green line to indicate when the colors are (closely) matching:

HLG

PQ

HLG matches on the bright saturated colors, while PQ matches on the darker colors. I suspect this happens because the 10-bit colors in the video frame are being cramped into a 8-bit color in the canvas. So we might need to wait for 10/16-bit color support in WebGL's texImage2D function in Chromium to resolve the black canvas and get perfectly matching colors.

The only thing I'm still wondering is if the matching colors change when you move the SDR content appearance slider in the Windows settings. I don't expect it to, because we are using the Rec2100 color gamut, but we are displaying 8-bit colors, so who knows...

[Askejm]

@WesselKroos I'm glad you found the pictures useful! They were fun to make.

As for the SDR brightness slider, it appears to have no effect to Chrome in HDR10 mode. Seems to be fixed at 50%? All other color spaces the SDR brightness slider will affect the brightness of the entire browser, including the HDR video.