Brainstorming alternatives

The following comes from an eatherpad created back in early 2012.

NB: there are two distinct use cases called "responsive":

[media] choosing different image based on media query for purpose of layout, e.g. portrait or landscape image depending on device orientation, or full-view or tightly cropped image depending on width of the viewport. The key is that it's an artistic choice, not merely a mechanical transformation of one image to another.

[dpi/bandwidth] choosing different representations of the same image. All variants have same aspect ratio, same content, and differ only by DPI and compression. These variants can be created automatically by the server (by downsampling or recompressing) and don't need human help.

Browsers have to conserve memory. Browser with low-res display may choose to load high-res assets to have nicer graphics when zoomed in, etc.

Those two use-cases are different, and both need a solution (a combination of two solutions probably).

---

Some possible solution age problem:

#1 -- NO GO [media]

<img src="mobile-size.png">
  <source src media>
  <source src media>
</img>

x Cons: In DOM, Chrome/Firefox lose child relationships of source and img relative to image. Confirmed by Paul Irish: at least in Chrome, “looking ahead” for sources on an / tag isn’t viable. Hixie said this is a lost cause. This cannot be backwards compatible because no browser can realistically update parsing. The "test case" presented by _crossdriver is flawed. It is just showing that renders in browsers. @media based rather than bandwidth-based Responsive features in legacy browser dependent upon a polyfill that would replace the initial img's src with an appropriate URL from the immediately following tags depending on their media query. Ouch.

Continuing work: _crossdiver wants to keep working on a polyfill for this, even though he'll be fighting the DOM. (c/o DOM Wranglers, Inc.)

#2 -- NO GO [media]

<image>
  <source src media>
  <source src media>
  <img src />
</image>

Cons: In DOM, Chrome/Firefox lose child relationships of source and img relative to image. Browsers may be using as an alias for to try to be "helpful" to careless HTML editors. I do believe this happens. Confirmed. the @media based selector breaks from the codec/file format selector currently used in and elements. To be honest we could also use @media selectors (and available bandwidth selectors) on and elements also. Why send a 720p video to a device that only supports 320p? means duplication of graphic-elements (there's AND then. both for same semantic meaning)

Pros: Adopts the same content model as other multimedia elements like and

#3 [media]

<picture alt="Alt tag should accurately describe the contents of all sources, though cropping and zooming may differ.">
  <source src media>
  <source src media>
  <img src />
</picture>

Cons Forging a new element. This is a con due people have to learn it. (Yeah, but it seems like a much smaller ask overall than “change the img tag” would be, y’know?) means duplication of graphic-elements (there's AND then. both for same semantic meaning) @media based rather than bandwidth-based. Semantics of “picture” usually imply photographs and are not as relatable to graphics, illustration, and other types of images.

Pros Previously suggested in 2007 http://lists.w3.org/Archives/Public/public-html/2007Jul/0121.html http://www.w3.org/Search/Mail/Public/search?type-index=public-html&index-type=t&keywords=picture+element http://lists.w3.org/Archives/Public/public-html/2007Jun/1057.html http://lists.w3.org/Archives/Public/public-html/2011May/0386.html http://www.brucelawson.co.uk/2010/why-video-audio-canvas-arent-self-closing-tags/ (the comments) Adopts the same content model as other multimedia elements like and Can adopt the same "media" attribute for source elements in and .

#3b [media]

This pattern could improve structure of alt (e.g allow marking up abbreviations in alt, or include entire dialog in alt of a comic strip):

<picture>
    <source media>
    <source media>
    <img src>
    alt text is <em>here</em>
</picture>

UA would not render and elements inside , all the other content is the fallback.

[this just leveraging the idea of HTML within being used as fallback content. Probable similarities to how alt gets used in

/ situations - nicolas]

Added 2012-03-01 by Mikko Rantalainen: it would make more sense to say that anything except is fallback / alternative content. As a result, you could put backwards compatible with an alt attribute there or your could do something else. Doing it this way would allow using as an alternative for the in the future, too.

#4 -- ATTRIBUTE PATTERNS [media]

<img src="http://cdn.url.com/img/myimage_xs.jpg" 
     media-xs="(min-device-width:320px and max-device-width:640px)" 
     media-src-xs="http://cdn.url.com/img/myimage_xs.jpg"  
     media-m="(min-device-width:640px and max-device-width:1024px)" 
     media-src-m="http://cdn.url.com/img/myimage_m.jpg"  
     media-xl="(min-device-width:1024px)" 
     media-src-xl="http://cdn.url.com/img/myimage_xsl.jpg" 
/>

Cons: Somewhat convoluted syntax, could be hard to learn/teach/document. Need of documentation AND limitation to syntax (only 3-5 variants possible: S-M-L-XL) lowsrc was previously dropped (this could also be a "pro", since it means re-adding is easier) And if the main case for dropping “lowsrc” was lack of use, it stands to reason that demand should bring it back. (Although that's less flexible than media query, since it's a fixed "low" definition.) - But lowsrc was not for viewport-dimensions but bandwidth concerns lowsrc is not the right way to go. As you said, it's less flexible. Hardcoding presentation in the structure. What if I was the medium image to show at min-device-width of 680px instead, recode all of the images?

Pros: less elements, slightly less typing Backwards compatible. No new elements. if we can find definitions (see cons *2) it is easy to adopt and don't need learning of a new element / structure in element.

#5 -- MEH

<object >


</object>

• Existing syntax seems very awkward here.
• Does anyone like using ? means duplication of graphic-elements (there's AND then. both for same semantical meaning just as it's been since html4)

#6 -- NO GO [media]

<figure>
        <source="bigimage.png" media="min-width: 600px">
        <img src="fallback.gif">
</figure>

Cons Completely redefines semantics and content model of an already new element. Current uses include multiple images, videos, canvas, code snippets, etc.

Pros: Maybe it helps that the element is somewhat in flux? I mean, messing with seems more taboo than something new.

#7 [media]

Existing idea from mailing list - http://lists.w3.org/Archives/Public/public-html/2011May/0386.html

<img src="fallback.jpg" alt="" srclist="myimagesrc">
<sourcelist id="myimagesrc">
  <source src="big.jpg" media="min-width: 600px" width="600" height="400">
  <source src="small.jpg" media="max-width: 600px" width="320" height="320">
</sourcelist>

Pros:

• Easy polyfill potential.
• Cleanly backward compatible.

Cons Same problem as any attempt to modify parsing. Basically, no worky. sourclist may not be close to img in DOM. (Is that good or bad?) It's a sibling to img not a child which is the wrong way when thinking about semantics because relation is not given immediately. needs an ID to identify AND an additional attribute to img-element to work.

#8 - [dpi/bandwidth]

<canvas data-src="mobile.jpg" data-fullimg="fullsize.jpg">
    <img src="mobile.jpg" alt="description of image" />
</canvas>

Pros: This pattern should theoretically work right out of the box; no fuss, no muss. All the responsive” logic is in the scripting that controls the canvas, and the img is provided as a no-JS/no canvas fallback.

Cons: JS is absolutely required for anything to work. We’re basically no better off with this pattern than finding an especially clever hackish workaround. With JS disabled, only Chrome shows the fallback (with JS on, it makes both requests)—it seems that’s the only browser that follows the spec. Several browsers are showing nothing at all—it’s possible this can be polyfilled, but the lack of no-JS support means it isn’t especially viable.

#9 - [dpi/bandwidth]

<img src="image.jpg" dpi="70 92 100 120">

The browser picks the best dpi and sends the request as:

image.jpg?dpi=70 (or perhaps some other url that doesn't require server side processing, but can be stored as a simple file, maybe image.jpg:dpi=70)

This is somewhat different use case from the previous ones, as this doesn't address artistic/layout purposes (e.g. landscape vsg portrait picture for screen rotation), but only optimisation of the same picture.

Pros:

Completely backwards compatible No new tags No server side code needed (if the url is chosen right). No javascript Browser stays in control of the size No fixed sizes, any number can go into the list.

Cons:

Magic URL. This can cause caching problems in browsers as it requires a ? in filename. Browser could use HTTP header instead of an URL, e.g. Accept-DPI: 120

#10 [dpi/bandwidth]

<img src="image.jpg" highsrc="image@2x.jpg">

A simplified version of #9. Image is either displayed 1:1 with CSS pixels (like desktops do) or at double that resolution ("Retina" display).

Browsers never supported real DPI for CSS pixels, and mobile pages are rarely viewed at 1:1 with device pixels because of zoom.

Mac OS X dropped support for arbitrary DPI and only supports 2x displays now. It's possible that evolution of displays will go in that direction.

The attribute could be called src2x="" in case we'll need src3x="" in the future.

Pros: much less verbosity in line with existing usage of "Retina" displays

Cons: Actual DPI won't be exact, like in current

Preparse reframe:

1. JavaScript can already make modifications to the DOM that cause lookahead parsers trouble. When this happens, there is a fairly stiff penalty associated with it.
2. JavaScript that fits the description of #1 isn’t going away.
3. Right now, browsers have no information about whether or not any piece of JavaScript will cause problems.
4. Having knowledgeable document authors identify for browsers which JavaScript will impact parsing and asset loading would give browsers the opportunity to handle that JavaScript differently in some fashion.

I’m not so bold as to say the correct thing for the browser to do would be to delay loading assets. Maybe browsers could figure out what speed connection the browser has and make a judgment call. Who knows.

I think time when assets are loaded should be left up to the UA. Specifically, UA should be allowed not to load the image at all if it's outside view (e.g. imagine an article with comment thread that has 100 avatars. None of those assets need to be loaded if user never scrolls down to comment section). Currently such optimisation is not possible for <img>, which blocks onload. This spec could fix that.

One of the things I like right now is that each browser vendor is experimenting with how to make things run faster. I see something like the preparse attribute as continuing that effort despite the fact that on the surface it may seem like the opposite of async and defer.

So maybe preparse is the wrong name. Instead, let’s simply ask browser makers, “If document authors are going to write scripts that they know will impact document parsing and asset loading (like Responsive Image JS does), would it be beneficial for you if document authors flagged the JavaScript accordingly?”

Given the emphasis on performance from browser makers right now, I believe that if we left it to the browser makers to figure out what to do with the information, that we would likely see them handle it better than our original ideas for the attribute.

2-part polyfill posibility: - NO GO

Assumption #1: I believe I saw somewhere (can't find the source anymore) that showed that a parent element that is hidden with display: none will prevent a child resource from being downloaded. That needs confirmation in all browsers. E.g. The asset in <img src="pic.jpg" style="display: none"> will still be downloaded. But this markup:

will prevent the asset download. (Hoping I didn't halucinate the blog post explaining this. Edit: I did halucinate. the blog post was referring to background images not inline images. doh!) Relevant resources: http://www.quirksmode.org/css/displayimg.html testcase http://www.phpied.com/smart-browsers-dont-download-unneeded-images/ (background images) http://www.thecssninja.com/css/even-better-image-preloading-with-css2#comment-2333 using generated content

Assumption #2: Also, I believe an inline JavaScript in the head will be run before any

content is parsed and assets pre-fetched because the browser will block for inline JS. This also needs confirmation in all browsers. Define "parsed" for this case. I believe the markup will be read, but the DOM construction will wait. Lots of cleverness in the timing of things here by browsers.

If both assumptions are true, couldn't we have a 2-part Polyfill solution for some of the above markup proposals?

One inline JS in the head that:

• does feature detection to ensure later JS will work
• Writes inline CSS that would match the parent items that appear later in the and applies display: none to those parent items.

Second JS that can be loaded later:

• figures out which src actually needs to be used and applies that to the appropriate child element (or writes a new element, whatever)
• unhides the parent elements

Current Polyfill Possibility #2 http://www.w3.org/community/respimg/common-questions-and-concerns/

One thing Scott Jehl mentioned to me (grigs) early on when I started testing his Responsive Images script was that there was different browser behavior when the script was inline versus external.

I sent a note to Yoav, who conducted the race conditions tests, to find out if he tested inline as well as external. It may be possible to get something like Responsive Images JS to work before asset loading if it is inline. Needs to be tested.

Yoav says inline scripts block (i.e. no race condition) on Chrome&FireFox. They do not block on IE9 (desktop, didn't test mobile): http://twitter.com/yoavweiss/statuses/141634994354393088

Probably not the best solution regardless because there is no contract between the browsers and authors that would prevent browsers from changing the behavior in the future.