[Feedback] Sustainability Feature Explainer

[tayloregivens]

This issue is meant to collect feedback about the proposed DevTools Sustainability Feature explainer that has been created based on the feature request from this thread

[fershad]

Great to see that there's a proposal to get this into DevTools!

A few comments and questions:

I notice the mention of web app a lot. I assume that's a generalisation for anything that's loaded in the browser (web page, SPA, MPA, PWA etc?)

---

Therefore, we will provide a table that shows the number of custom fonts, large photos, and other properties in a color-coded manner to indicate which counts are above or below average to give developers a sense of how sustainable their website is compared to other sites.

“Client-Device Energy Score“ is out of 100 and is based on the CPU utilization and GPU utilization of the web app's initialization compared to the average web app.

What averages would you be looking to use for these?

---

Use an open-source API, such as the [Green Web Foundation API] (https://www.thegreenwebfoundation.org/green-web-feed/), to estimate carbon emissions due to the energy required to transfer bytes over the network

The Green Web Foundation have a library called CO2.js which would probably be useable here. It currently uses the Sustainable Web Design model you've mentioned in the explainer to generated a full system carbon emissions estimate - with & without the caching & return visitor assumptions the model makes. That said, it shouldn't be too hard to update it to pull out just the emissions from network data transfer alone.

---

For the Energy Consumption section, would it not be possible to use the usage data from CPU & GPU, and use that with a CO2 g/kWh figure to generate an estimate for carbon emissions for loading a web app? There are a few providers that can provide country/region level emissions intensity data. Ember has a pretty comprehensive data set with average yearly grid intensity. The "world" figure of 442g/kWh could be used as a fallback.

Right now, being able to get emissions estimates at the device level is very difficult. If that can be surfaced in DevTools it to be a fantastic feature, and extremely helpful in giving a more complete picture of the overall sustainability profile for a website/web app.

[radum]

I agree its amazing to see these new developments but I worry we are rushing into it and risk providing false information to devs.

estimate of carbon emitted from the network transfer energy that goes into rendering a web app

How exactly does that work? Because this is a highly contested affirmation. It would be extremely helpful if you could link any research papers that backs that up for us to see that we are not just marketing our climate problems.

[hxlnt]

@radum Definitely hear you on the nuance of the calculations and the risk of misleading or misrepresenting the impact. On the other hand, it feels like providing some assessment, even if imperfect, is the right first step toward growing both developer awareness and the assessments themselves. So we're hoping to strike the right balance there.

The explainer and @fershad's comment above refer to existing APIs and projects that have made their calculation approaches available at the links provided. I would expect anything we include in DevTools would build on existing expertise like this rather than trying to reinvent a calculation from scratch.

Appreciate the feedback; this is exactly why we want to collaborate on this approach in the open and drive toward an outcome that makes sense and is meaningful rather than simply greenwash-y. 👍

[radum]

Thanks for your reply @hxlnt. I agree with your position and I understand that there has to be a start. Unfortunately I disagree with the approach we are taking as a community. Although it seems simple and we try to narrow to reach IMHO this has bigger ramifications on the long term. All those mentioned about fundamentally have the same baseline where amount of data over the wire is quantified into amount of CO2 emissions. I believe and this has been contested lots that this is wrong. It is almost impossible to make that correlation.

Also raising awareness for devs is amazing and there are so many things we can do, but at the end of the day, each developer earns his keep with his trade, and what they do will eventually be translated in some form to payed work for clients. If we make the tool happy we show the paying customer we achieved what we were set to do and in loads of cases the story ends there. And this becomes a marketing tool.

Sustainability and CO2 emissions are 2 different things. Although they overlap indirectly in the grand scheme of things they have different implications.

Don't get me wrong I am not in the climate denialism team and I have been involved in this for a very long time, but I just want to make sure people will do more in those areas with meaningful impact. I don't believe that the front facing side of a website can be quantified in terms of CO2 emissions.

All the nuts and bolts behind it that makes it run, content generation, the way it is built, deployed, tested and so much more yes, maybe, there are ways. But to say that www.microsoft.com when is accessed by a user generates xx CO2 and if there are millions of users that access that site each day we multiply that number is just wrong not imperfect.

I get then enthusiasm and the energy some of us have to do more with the skills we have and I love it, but we also have to understand our limits around what we know.

I would love to see this in Dev tools with information around the energy impact and so on but the correlation to CO2 emissions IMHO has a negative impact down the chain.

[ldevernay]

I agree that this feather would be awesome.
However, some points (most of them have been already highlighted) need some attention.
Tagging issues related to sustainability is great but it raises a question : how are other issues related to sustainability? Are they not impactful or could they decrease sustainability? Or maybe we don't know? I think it's all a matter of presentation : the implication of tagged issues on sustainability is known and documented (but, some other issues might later be proven as related to sustainability)
Evaluating CO2 emissions is a problem. The actual formulas used for this are debatable but displaying this metric could prove useful to conduct continuous improvement. The matter is not to have something as precise as possible but to get reference to get better, as detailed by Ismael Velasco : https://ismaelvelasco.dev/emissions-in-1gb
But I think the real matter is elsewhere : focusing on CO2 emissions can prove counter-productive.

1. This should not be CO2 but CO2 equivalent emissions because CO2 is not the only GHG to take into account.
2. Keeping only one environmental metric could lead to pollution transfer. One quick controversial example : if we only look at GHG emissions, nuclear energy is obviously the way to go. But there are other factors to take into account, such as hydric stress, abiotic resource consumption and disposal, etc. Regarding digital services, most of the impact comes from the manufacture of user devices. As such, water consumption (which is also proven problematic regarding data centers), consumption of abiotic resources (hugely needed to manufacture devices and causes lots of ethical and environmental concerns) and preservation of biodiversity could also be taken into account. It obviously means additional formulas, each with its own range of assumptions, hypothesis and biases. But relying only on CO2 is flawed.
3. This might lead some people into thinking that carbon offsetting could be THE solution, even it is only a part of the solution and sobriety (then efficiency) should be a priority.

On DevTools, one will mostly measure efficiency (which is a part of sustainability) but sobriety should also be mentioned (even if it goes as far as UX/UI... and ideally business model, content & data management, for example).

Regarding the issues tagged with Sustainability, on which list/repository are they based? Where/how will you gather thresholds (could a page weight above 1MB or, even better, 500kB be noted as an issue?), references and other resources to document this tab?

So yes, I really want to see this Sustainability tab appear and would love to help make it happen in any possible way. The road ahead is full of tricks and blocks, let me know if I can be of any help.

[davidmytton]

Great to see this work - providing metrics to developers through the browser devtools is a good way to surface information to the people who can make efficiency improvements. This is the subject of my research at the University of Oxford, so I'm pleased to see how it can be applied to the real-world.

My main comment on this proposal is that data transfer is not proportional to energy consumption and the kWh/GB metric used in most website calculations is wrong for the purpose it is being used. It is not accurate for estimating the energy consumption over the network and is useful primarily for historical reporting of energy intensities where the total energy consumption and total data transfer over a given network are already known. This is backed by several papers including Malmodin, 2020, Carbon Trust, 2021, and Koomey & Masanet, 2021. I discussed this in Mytton & Ashtine, 2022, a blog post, and have an upcoming paper going into detail on the topic.

I'd also comment that "carbon emissions" is not a very useful metric to expose because it requires temporal and spatial context that has so much variability as to make it difficult for developers to optimize. The value can range significantly depending on the time of day and location, and will trend down over time as grids decarbonize. It'll be different for each component of the system - the local browser, network, and wherever the site is hosted - and have the same variability at each point. I'd suggest that exposing energy metrics would be much more consistent and actionable.

Firefox 104 has taken a good step towards this by introducing a new power profiler for Windows (Windows 11 or Windows 10 with an EMI) and Mac (Apple Silicon only). This provides output in watts in the profiler UI, but the internals are measuring in picowatt-hour. On Mac it's using the task_info API. This also has the advantage of being directly measurable from the browser, so can be optimized.

After that, the Google Lighthouse metrics are probably the most useful because they consider all the other aspects mentioned in the proposal e.g. caching, unused CSS, number of requests, etc. These are often ignored by simple website calculators, which mean that real optimizations such as using a CDN, adjusting caching headers, using different image formats etc, are invisible.

Providing an assessment is the right goal, but improvements must be able to be reflected in the scoring. Local power consumption is where I'd start. Happy to discuss further / comment on further iterations of this proposal.

[mgifford]

It would be really good to be able to launch this from the command line so that we'd be able to evaluate a list of pages and extract just the total energy used to load and render each of the pages in the list. Page by page assessments are useful if you are looking to profile specific code. However, a CLI interface would allow for testing other things than simply the energy used in different versions of a page. Automation is key to broader adoption.