Intrinsic dimensions

[yoavweiss]

While working on yoavweiss@d0fda08 to fix #82 I realized that the browser doesn't have enough data with current syntax to get the intrinsic dimensions of the image, only its intrinsic width. The height remains a mystery until the image is downloaded (or until Layout says otherwise).

AFAICT, in order to resolve it, we need to add some data in the syntax regarding height. I propose the following:

• Add an 'h' descriptor.
• If authors want the browser to avoid re-layout after image download (assuming CSS doesn't enforce image dimensions), they needs to provide both 'w' and 'h' to at least one of the resources in each source list (so in each srcset attr)
• The proportion between the first/last resource dimensions that include both 'w' and 'h' will be used to calculate intrinsic dimensions of the picked resource (even if it doesn't include both 'w' and 'h')

Now, I might have missed something, or the above proposal may be bad. Let me know if you think that's the case.

[eeeps]

I don't want to derail the conversation from the problem you're trying to address (re-layouts... for which an 'h' descriptor sounds like a pretty nice solution). But...

Edit: snip, as I then went on to derail the discussion (: Moved to #86.

As an author, being able to add an optional 'h' to one of the sources in order to define the aspect ratio and prevent reflows sounds like an easy win. So this SGTM.

[yoavweiss]

@eeeps - I think this is an important discussion, but can you please open a new issue regarding that? (since it's a separate subject from intrinsic dimensions calculation). It'd enable to keep both topics' discussions on subject.

Thanks :)

[igrigorik]

Add an 'h' descriptor.

srcset does provide height descriptor [1]. That said, one of my takeaways from the resp-img meetup back in September was that h was a very likely candidate to be dropped from the srcset spec. Multiple people (e.g. Boris, who implemented the srcset polyfill) pointed out that the selection algorithm when both w and h are present is a nightmare and is near impossible to figure out. Similarly, when polling the room, we couldn't come up with any compelling use cases for h alone.

But even with that aside, isn't the intent of srcset w and h descriptors to capture "maximum viewport width/height", not describe intrinsic image dimensions? In other words, this seems like the wrong mechanism to begin with... or am I missing something?

[1] http://www.w3.org/html/wg/drafts/srcset/w3c-srcset/#additions-to-the-img-element

[tabatkins]

@igrigorik Don't be confused by the surface similarity with HTML's srcset syntax - the (unimplemented by anyone) w/h descriptors it uses there are compact ways of writing MQs. In <picture>, the w unit (and potentially an h unit) are the physical sizes of the image source, in image pixels. Completely different things.

[igrigorik]

@tabatkins ah, I see... Indeed, didn't catch that. Knowing that, the examples make a lot more sense - #5 in particular. FWIW, it may be worth calling out this difference more explicitly in the spec.

Since I'm two steps behind, let me see if I grok the proposal via an explicit example:

  <picture>
    <source sizes="(max-width: 30em) 100%, (max-width: 50em) 50%, calc(33% - 100px)"
            srcset="pic400.jpg 400w 200h, pic800.jpg 800w, pic1200.jpg 1200w">
    <img src="pic400.jpg" alt="The president giving an award.">
  </picture>

Let's say my device max-width is 30em (with 16px default font size) and my DPR is 2.0, then:

• image is intended to be displayed at 100% viewport width
• 30em * 16px * 2DPR = 960px image (ideally)
• UA downloads pic1200.jpg (and downscales on the client)
• UA uses 200h to infer the ratio and (unless CSS overrides it) reserves 480x240px box for the image

Does that look about right?

P.S. Still need to grok how this would also work with sizes + w/h (ala #86).

[tabatkins]

Yup, that's exactly right.

[zcorpan]

Do we need this? Traditionally you would just use width/height attributes on the img element. Is that not good enough?

[zcorpan]

Hmm. Nevermind, that doesn't work for art direction.

[yoavweiss]

Yeah. This bug would be resolved with @tabatkins's proposal

[zcorpan]

I think it is slightly cryptic that a hight descriptor on one image source magically sets the aspect ratio for other image sources also. I don't think that is especially clear for authors. From the example above, it looks to me like the height descriptor is missing from some of the sources. Moreover, the calculated aspect ratio might not exactly match the other image's actual aspect ratios, due to rounding, resulting in the intrinsic height being slightly off.

I think if we want to specify the aspect ratio for all the sources, it would be clearer to have a separate attribute (still subject to rounding). Or use the height descriptor for each image source individually.

[yoavweiss]

Mandating authors to specify height on every image can become cumbersome quick.
Deriving aspect ratio from the first (height, width) tuple is terser, but (as you say) that's not always an advantage...

All in all, I think the best would be to conclude ratio from first tuple, and override it with more specific ones if provided by author, but since it's a matter of authoring convenience (and I'm far from being an expert on the subject or a day-to-day author), I'm not hung up on any particular syntax as long as the browser can conclude the intrinsic size.

[Wilto]

Alright, after a bunch of discussion in the IRC channel, it’s looking like the prevailing opinion is this: for situations where the aspect ratio of all sources in a source will match (+/- rounding errors), <source aspect[-]ratio="16/9" sizes="100%; small.jpg 400, med.jpg 800, big.jpg 1600"> has us covered, with the new attribute using values from http://www.w3.org/TR/css3-mediaqueries/#aspect-ratio

In the event that the sources have differing aspect ratios, adding in-syntax overrides gets clunky in a hurry. We have a handful of votes for relegating differing aspect ratios to different source elements, since it arguably falls under the “art direction” case anyway—it’s not as terse, but this is sort of a special case.

[aarongustafson]

100% agreed on that approach.

On Wed, Feb 26, 2014 at 1:17 PM, Mat Marquis notifications@github.comwrote:

Alright, after a bunch of discussion in the IRC channel, it’s looking like
the prevailing opinion is this: for situations where the aspect ratio of
all sources in a source will match (+/- rounding errors), <source
aspect[-]ratio="16/9" sizes="100%; small.jpg 400, med.jpg 800, big.jpg
1600"> has us covered, with the new attribute using values from
http://www.w3.org/TR/css3-mediaqueries/#aspect-ratio

In the event that the sources have differing aspect ratios, adding
in-syntax overrides gets clunky in a hurry. We have a handful of votes for
relegating differing aspect ratios to different source elements, since it
arguably falls under the “art direction” case anyway—it’s not as terse, but
this is sort of a special case anyway.

—
Reply to this email directly or view it on GitHubhttps://github.com//issues/85#issuecomment-36157353
.

[zcorpan]

LGTM. The dash should be omitted in the attribute name to match HTML's naming convention.

If the value is invalid, the attribute should just be ignored.

[zcorpan]

Hmm, thinking about this some more, this only helps when using sizes and width descriptors. If you use density descriptors instead, there's no way to set the intrinsic width/height of the source. We could add width/height attributes for that case maybe?

[eeeps]

Count me as slightly opposed, especially if/when #86 is addressed and there's a second reason for h descriptors to be a thing. I expect I'd be doing a lot of this...

<picture>
  <source
    sizes="contain"
    srcset="large.jpg 1024w 768h,
            small.jpg 512w  384h"
    aspectratio="1024/768" />
  <img src="small.jpg" alt="¡Obama mambo!" />
</picture>

...just copying over an aspectratio from the highest-res source's dimensions. In the absence of an aspectratio, could we check if any source in the srcset has both h and w specified, to save a bit of duplication?

[eeeps]

If you use density descriptors instead, there's no way to set the intrinsic width/height of the source.

Why not

<picture>
  <source srcset="small.jpg 512w 384h 1x, large.jpg 1024w 768h 2x" />
  <img src="small.jpg" alt="Biden doing the shim sham" />
</picture>

It feels weird to me to set an intrinsic width and height on a <source> that lists multiple files, each having different dimensions. Intrinsic dimensions should be set on a per-file level.

[nwtn]

I also dislike setting width and height on source, I think it opens up a lot of possible confusion. If it came down to width and height on source vs a reflow, I'd probably vote for the reflow.

@eeeps I think if you're providing both w and h in srcset, there would be no need to provide aspectratio; it would only be needed if there wasn't another mechanism for the browser to determine intrinsic width and height.

[yoavweiss]

I also dislike setting width and height on source, I think it opens up a lot of possible confusion

Confusion how?

[nwtn]

@yoavweiss because people are used to width and height attributes reflecting real width and height. usually display width/height, but sometimes this is the same as intrinsic width/height. if used with source here, they would definitely not be display width/height and would only be intrinsic width/height for at most one of the files. i think it could be confusing for developers to understand what values they should put there and why.

if we used different attribute names, i'd feel a lot more comfortable with it. i just think width and height come with a lot of cognitive baggage.

[zcorpan]

The intrinsic width and height are the width and height that are used for display (unless overridden by CSS).

In @eeeps 's example above, the width/height used for rendering for either image would be 512x368 CSS pixels.

I can buy that aspectratio/width/height attributes are more confusing or clunky than width/height descriptors, sure.

[nwtn]

It seems like there are tradeoffs with each option, but here's where I think we're at:

1. Use aspectratio for sources with sizes/w, and some other width/height attrs for source without sizes/w. The width/height attrs could be width/height, but I would prefer something without the baggage, like w/h. Disadvantage is that it's messy to have different attributes to handle this problem.
2. Use w and h descriptors for both types of source, as @eeeps suggested. Disadvantages are more work for the author to figure out and maintain all the w/h values, and messier srcset syntax.
3. Use aspectratio attr for both types of source and use the width/height values provided there as the intrinsic width and height for the non-sizes source. Disadvantage is that it's confusing to have an attribute called "ratio" that requires literal values (e.g. aspectratio="200/100" would not behave the same as aspectratio="2/1").
4. Use width and height attributes (either width/height or w/h) for both types of source. Disadvantage is, as @eeeps pointed out, for the sizes case you're duplicating values .

I think that about covers it, but tell me if I'm wrong. Given these options, from an author POV, I would prefer 1 because it involves the least amount of duplication and figuring-stuff-out, keeps srcset relatively clean, and isn't all that confusing.

But I am but one man...

[zcorpan]

That's about right.

I think most authors are not going to understand the details here when they write their markup. So that suggests that we should make it hard to do something that doesn't make sense. Having more attributes to choose from gives more opportunities for doing something nonsensical (such as including all attributes and all descriptors, because, hey, they're available, that means they have to be filled in, right?). For this reason I'm leaning towards (2). Note that it doesn't have to be more work for the author, it's enough to specify w/h for one of the image sources.

[eeeps]

I think most authors are not going to understand the details here when they write their markup.

Having just misunderstood the details myself, I agree! I wasn't clear until reading @zcorpan's comment that intrinsic dimensions and resource dimensions were different things.

Frankly I stopped using width="" and height="" on <img> a long time ago because I couldn't figure out what they meant in the context of a responsive design (where an image's display dimensions are usually a function of the layout, rather than a function of the resource dimensions). Were height="" and width="" the resource dimensions or the display dimensions? If the former, what do they do? If the latter, dosen't display info belong in CSS?

Reading up, I think I understand now that "intrinsic dimensions" are the display dimensions before CSS.

• They can be specified by the author in height="" and width="" attributes.
• If they aren't specified by the author they're considered to be zero until the resource has loaded, and then they're inferred from the resource dimensions (and any resolution information in markup, defaulting to 1x).
• They establish a baseline geometry in CSS pixels (height/width/aspect-ratio) which the browser layers any CSS sizing on top of to figure out the final layout dimensions.
• While the CSS and image resource are loading, the browser uses intrinsic dimensions to reserve space for the image.

Practically, for authors, the only reason to specify intrinsic dimensions in markup rather than have the browser infer them from the resource dimensions once the resource has loaded (as is my habit) is to prevent reflows that might occur when image resources finish loading after the CSS.

Do I have all that right?

If so, I agree with @zcorpan on (2). The whole concept of specifying intrinsic height, width, and/or aspect ratio separately from resource dimensions is very confusing — better to only have to worry about the latter.

Not sure how #107 figures in with this though?

[zcorpan]

It's probably a good idea to fix this at the same time as fixing #86 so that we don't introduce a syntax that does one thing and later reuse that syntax to also do something else.

[aFarkas]

I actually like the idea of having an attribute called aspectratio on img/source. In case a developer has defined w and h descriptors for at least one candidate the aspectratio is defined implicit.

I also don't think, that this attribute doesn't work with x descriptors. While there are possibilities to define this with markup. We simply need either the width or the height. And for this we could re-use the width declared on the img (means one width for all sources) or re-use the sizes declared on source (i.e: sizes="300px", yes sizes in conjunction with x descriptor ;-)) or could re-invent a new width attribute on the source element.

But for the actual practice, this isn't important. What is actually important is that the attribute works in conjunction with CSS, where the author can declare a specific CSS side (mostly width) and the corresponding other side (height) is automatically calculated (as long as the CSS value is either initial/auto), so that the browser can occupy the space before the image is loaded:

<img 
    style="width: 300px; height: auto;"
    aspectratio="3" 
    srcset="600x200.jpg 2x" src="300x100.jpg" />

This really means we only need an aspectratio attribute, nothing more.

[zcorpan]

In my opinion we should do what has the most chance of being used correctly most of the time even when the Web developer doesn't understand the details. In your example you don't need the aspectratio attribute at all, you can just say height: 100px.

[aFarkas]

@zcorpan
Yeah, this is true. But only wanted to make an easy example how this should be calculated/interact with CSS.

Here are 3 different examples:

<picture>
   <source 
    srcset="200x300.jpg 1x, 400x600.jpg 2x"
    aspectratio="0.666" />
   <img 
    style="width: 300px; height: auto;"
    aspectratio="3" 
    srcset="600x200.jpg 2x" src="300x100.jpg" />
</picture>

<img
  style="width: 100%; height: auto;"
  aspectratio="2"
  srcset="300x150.jpg 300w, 600x300 600w" />

<img
  style="width: 100%; height: auto;"
  srcset="300x150.jpg 300w 150h, 600x300 300h 600w" />

And I think this is quite easy. The developer will know what aspectratio means and at least if one side is defined the other one can be calculated.

In case an x descriptor is used and both sides are auto/initial the dimension can't be calculated, but I'm fine with this.

[nemzes]

IMHO, there's one more reason to not rely on the h descriptor within source for this: the fact that it makes it easier to misunderstand the purpose of each descriptor. If I have the h option then I might be tempted to write this:

<img srcset="small.jpg 320w 160h, large.jpg 640w 640h" />

When what we want is to (gently) push authors to write this:

<picture>
    <source media="(max-width: 320px)" srcset="small.jpg" />
    <source srcset="large.jpg" />
</picture>

So, it makes it easier to get the art direction concern bleed into the wrong feature. Using aspectratio makes the separation of concerns clear:

<picture>
    <source aspectratio="2/1" media="(max-width: 320px)" srcset="small.jpg" />
    <source aspectratio="1/1" srcset="large.jpg" />
</picture>

If you try to use aspectratio with the sources of the <img> example above (instead of h) it becomes immediately obvious it's not the right solution. Since an author will most likely trying to be solving an issue at a time (e.g. "make the browser allocate space for the image"; "load the correct art-directed image") it's probably a good thing that misuse of a feature will force them to look up the correct solution.

[zcorpan]

This issue came up recently on www-style:
http://www.w3.org/mid/CADBHO9Gcoujsk9OWGb0YZVZYxSwpdDFbB66UEG4N19YJ3jKG0g@mail.gmail.com

cc @OliverJAsh @rachelnabors @una @AmeliaBR

[zcorpan]

https://lists.w3.org/Archives/Public/public-respimg/2016Nov/0000.html (thread continues in December)

[MajPay]

I am a little disappointed this is still a thing, wanted to replace an old responsive-lazy-image implementation i wrote with a standard loading="lazy" method, just to find out it cant accomplish different aspect ratios.

Why cant it simply be like this:

<picture>
    <source media="(min-width: 640px)" srcset="large-src.jpg" width="600" height="300" />
    <source srcset="small-src.jpg" width="400" height="300" />
    <img src="large-src.jpg" loading="lazy" width="400" height="300" />
</picture>

So the matched source would be able to override the existing widht/height attribute?

I cant see any problem with that and it would be much more intuitive to use.

Should @container-queries be taken into account as well?

<picture>
    <source container="(min-width: 640px)" media="(min-width: 640px)" srcset="large-src.jpg" width="600" height="300" />
    <source srcset="small-src.jpg" width="400" height="300" />
    <img src="large-src.jpg" loading="lazy" width="400" height="300" />
</picture>

Please make this happen so i can remove all my custom implementations that are a real p*** in my a** to maintain.

[yoavweiss]

I am a little disappointed this is still a thing, wanted to replace an old responsive-lazy-image implementation i wrote with a standard loading="lazy" method, just to find out it cant accomplish different aspect ratios.

Why cant it simply be like this:

<picture>
    <source media="(min-width: 640px)" srcset="large-src.jpg" width="600" height="300" />
    <source srcset="small-src.jpg" width="400" height="300" />
    <img src="large-src.jpg" loading="lazy" width="400" height="300" />
</picture>

So the matched source would be able to override the existing widht/height attribute?

See whatwg/html@c2d1a36

[MajPay]

Awesome!

Chromium already returns "number" on this assertions: typeof document.createElement('source').width

Firefox seems to lack support for width/height as well as aspectRatio though...

Looking forward to this, thanks for your reply.