Skip to content

FlorentHaffner/electricitymap-contrib

BranchesTags
 
 

Repository files navigation

electricityMap Slack Status CircleCI Twitter Follow

A real-time visualisation of the Greenhouse Gas (in terms of CO2 equivalent) footprint of electricity consumption built with d3.js and mapbox GL, maintained by Tomorrow. Try it out at http://www.electricitymap.org, or download the app:

Get it on Google Play Get it on the Apple Store

image

You can contribute by

You can also see a list of missing data displayed as warnings in the developer console, or question marks in the country panel:

image

Check the contributing section for more details. Join us on Slack if you wish to discuss development or need help to get started.

Tomorrow is hiring!

The company behind the electricityMap builds tech to empower organisations and individuals to understand and reduce their carbon footprint.

We're hiring great people to join our team in Copenhagen. Head over to our jobs page if you want to help out!

Frequently asked questions

How do you define real-time data? Real-time data is defined as a data source with an hourly (or better) frequency, delayed by less than 2hrs. It should provide a breakdown by generation type. Often fossil fuel generation (coal/gas/oil) is combined under a single heading like 'thermal' or 'conventional', this is not a problem.

Why do you calculate the carbon intensity of consumption? In short, citizens should not be responsible for the emissions associated with all the products they export, but only for what they consume. Consumption-based accounting (CBA) is a very important aspect of climate policy, and allows to assign responsibility to consumers instead of producers. Furthermore, this method is robust to governments relocating dirty production to neighbouring countries in order to green their image while still importing from it. We published our methodology here.

Why don't you show emissions per capita? A country that has few inhabitants but a lot of factories will appear high on CO2/capita. This means you can "trick" the numbers by moving your factory abroad and import the produced good instead of the electricity itself. That country now has a low CO2/capita number because we only count CO2 for electricity (not for imported/exported goods). The CO2/capita metric, by involving the size of the population, and by not integrating all CO2 emission sources, is thus an incomplete metric. CO2 intensity on the other hand only describes where is the best place to put that factory (and when it is best to use electricity), enabling proper decisions.

CO2 emission factors look high — what do they cover exactly? The carbon intensity of each type of power plant takes into account emissions arising from the whole life cycle of the plant (construction, fuel production, operational emissions, and decommissioning).

Is delayed data useful? While the map relies on having real-time data to work it's still useful to collect data from days/months past. This older data can be used to show past emissions and build up a better dataset. So if there's an hourly data source that lags several days behind you can still build a parser for it.

Can scheduled/assumed generation data be used? The electricityMap doesn't use scheduled generation data or make assumptions about unknown fuel mixes. This is to avoid introducing extra uncertainty into emissions calculations.

Can areas other than countries be shown? Yes providing there is a valid GeoJSON geometry (or another format that can be converted) for the area. As an example we already split several countries into states and grid regions.

How can I get access to historical data or the live API? All this and more can be found here.

Data sources

Carbon intensity calculation and data source

The carbon intensity of each country is measured from the perspective of a consumer. It represents the greenhouse gas footprint of 1 kWh consumed inside a given country. The footprint is measured in gCO2eq (grams CO2 equivalent), meaning each greenhouse gas is converted to its CO2 equivalent in terms of global warming potential over 100 year (for instance, 1 gram of methane emitted has the same global warming impact during 100 years as ~34 grams of CO2 over the same period).

The carbon intensity of each type of power plant takes into account emissions arising from the whole life cycle of the plant (construction, fuel production, operational emissions, and decommissioning). Carbon-intensity factors used in the map are detailed in co2eq_parameters.json. These numbers come mostly from the following scientific peer reviewed literature: IPCC (2014) Fifth Assessment Report is used as reference in most instances (see a summary in the wikipedia entry)

Each country has a CO2 mass flow that depends on neighbouring countries. In order to determine the carbon footprint of each country, the set of coupled CO2 mass flow balance equations of each countries must be solved simultaneously. This is done by solving the linear system of equations defining the network of greenhouse gas exchanges. Take a look at this notebook for a deeper explanation. We also published our methodology here.

Real-time electricity data sources

Real-time electricity data is obtained using parsers.  

Click to see the full list of sources

Production capacity data sources

Production capacities are centralized in the zones.json file. Values in the capacity maps are in MW.  

Click to see the full list of sources

Cross-border transmission capacity data sources

Cross-border transmission capacities between the zones are centralized in the exchanges.json file. Values in the capacity maps are in MW.  

Click to see the full list of sources

A ⇄ B: bidirectional operation, with power flow either "from A to B" or "from B to A"

A ⇉ B: unidirectional operation, only with power flow "from A to B"  

Electricity prices (day-ahead) data sources

Real-time weather data sources

We use the US National Weather Service's Global Forecast System (GFS)'s GFS 0.25 Degree Hourly data. Forecasts are made every 6 hours, with a 1 hour time step. The values extracted are wind speed and direction at 10m altitude, and ground solar irradiance (DSWRF - Downward Short-Wave Radiation Flux), which takes into account cloud coverage. In order to obtain an estimate of those values at current time, an interpolation is made between two forecasts (the one at the beginning of the hour, and the one at the end of the hour).

Topology data

We use the Natural Earth Data Cultural Vectors country subdivisions (map admin subunits).

Contribute

Want to help? First of all, we'd like to invite you to join us on slack at http://slack.tmrow.co. On Slack you can see what we are working on and get in touch with other contributors. Also, if you get stuck, you can write a message in our channel #electricitymap.

There are different ways how you can help improve electricityMap, have a look at the overview above.

Steps to making a code contribution

Follow these steps to make your contribution:

  1. Fork the repository.
  2. Clone your fork of the repository.
  3. Download Docker.
  4. Set up and start your local environment (see explanation below).
  5. Make your code changes and test them in your local environment.
  6. Push your changes to your fork.
  7. Submit a pull request to bring your contribution into the production version.

Set up and start your local environment

Note: to test parsers, head directly to Testing parsers locally

The frontend will need compiling. In order to do this, open a terminal in the root directory and run

docker-compose build

One you are done, you can start the application by running

docker-compose up

This will watch over source file changes, run nonstop and watch changes you make in the code to recompile the frontend if needed. Note this only builds for the English language (which will be faster as not all languages need to be built). If you want to build all languages, change the command of the web-watch-en section of docker-compose.yml from command: npm run watch-en to command: npm run watch.

Head over to http://localhost:8000/ and you should see the map! Note that the backend is responsible for the calculation of carbon emissions. The map data displayed comes from a mock server providing dummy data from state file.

If you have issues building the map locally check out the Troubleshooting section below for common problems and fixes.

Logger

If you want to add new or change existing parsers you can use our public logger to see if your parser(s) can fetch data successfully. The logs show warnings and errors for all parsers.

Updating region capacities

If you want to update or add production capacities for a region then head over to the zones file and make any changes needed to the capacity map. Values are in MW. The zones use ISO 3166-1 codes as identifiers, a list of which can be found here.

Adding a new region

As a first step, do a search for the region on our GitHub, as contributors may have explored things before. It is very simple to add a new country. The electricityMap backend runs a list of so-called parsers every 5min. Those parsers are responsible for fetching the generation mix of a given country (check out the existing list in the parsers directory, or look at the work in progress).

I have not found a new data source You can help us by opening a new issue for the relevant region and add the relevant individual or organization to contact (ideally with their contact details on email, Twitter and phone). Doing so will ensure that we and other visitors interested can contact them to make them aware of the electricityMap. Usually, energy agencies, governments, transmission system operators are good potential sources. If you can't code, this is an amazing way to help us!

I have found a data source and I could help build a parser A parser is a python3 script that is expected to define the method fetch_production which returns the production mix at current time, in the format:

def fetch_production(zone_key='FR', session=None, target_datetime=None, logger=None):
    return {
      'zoneKey': 'FR',
      'datetime': '2017-01-01T00:00:00Z',
      'production': {
          'biomass': 0.0,
          'coal': 0.0,
          'gas': 0.0,
          'hydro': 0.0,
          'nuclear': None,
          'oil': 0.0,
          'solar': 0.0,
          'wind': 0.0,
          'geothermal': 0.0,
          'unknown': 0.0
      },
      'storage': {
          'hydro': -10.0,
      },
      'source': 'mysource.com'
    }

The session object is a python request session that you can re-use to make HTTP requests.

target_datetime is used to fetch historical data (when available). logger is a logging.Logger whose output is publicly available so that everyone can monitor correct functioning of the parsers.

The production values should never be negative. Use None, or omit the key if a specific production mode is not known. Storage values can be both positive (when storing energy) or negative (when the storage is emptied).

The parser can also return an array of objects if multiple time values can be fetched. The backend will automatically update past values properly.

Once you're done, add your parser to the zones.json and exchanges.json configuration files. Finally update the real-time sources.

Run all of the parser tests with the following command from the root directory:

python -m unittest discover parsers/test/

For more info, check out the example parser or browse existing parsers.

Generating a new map

If your changes involve altering the way countries are displayed on the map a new world.json will need to be generated. Make sure you're in the root directory then run the following command:

docker-compose run --rm web ./topogen.sh

For a more detailed explanation of how the map is generated see here.

Testing parsers locally

In order to test your parser, make sure first that you have installed the required modules as described, and are using python 3.6 (consider also using a virtual environment) in parsers/requirements.txt: for that you can run

pip install -r parsers/requirements.txt

From the root folder, use the test_parser.py command line utility:

python test_parser.py FR price  # get latest price parser for France
python test_parser.py FR  # defaults to production if no data type is given
# test a specific datetime (parser needs to be able to fetch past datetimes)
python test_parser.py DE --target_datetime 2018-01-01T08:00

Many of the tests require API keys of the data or web service providers and will therefore fail with an error message like

Exception: No ENTSOE_TOKEN found! Please add it into secrets.env!

In such cases, please browse the website related to the provider and ask for an API key. As soon as you got hold of the API key, make it an environment variable. This fixes the error.

Updating the map

We've added a testing server locally.

To add a new country to the map, run:

PYTHONPATH=. python mockserver/update_state.py <zone_name>

from the root directory, replacing <zone_name> by the zone identifier of the parser you want to test. This will fetch production and exchanges and assign it a random carbon intensity value. It should appear on the map as you refresh your local browser.

Troubleshooting

  • ERROR: for X Cannot create container for service X: Invalid bind mount spec "<path>": Invalid volume specification: '<volume spec>'. If you get this error after running docker-compose up on Windows, you should tell docker-compose to properly understand Windows paths by setting the environment variable COMPOSE_CONVERT_WINDOWS_PATHS to 0 by running setx COMPOSE_CONVERT_WINDOWS_PATHS 0. You will also need a recent version of docker-compose. We have successfully seen this fix work with v1.13.0-rc4. More info here: docker/compose#4274.

  • No website found at http://localhost:8000: This can happen if you're running Docker in a virtual machine. Find out docker's IP using docker-machine ip default, and replace localhost by your Docker IP when connecting.

Windows Specific

  • FATAL ERROR: Ineffective mark-compacts near heap limit Allocation failed - JavaScript heap out of memory. You can configure the memory allocation using NODE_OPTIONS=--max-old-space-size=4096 in the "build-release" script in package.json, example: export NODE_OPTIONS=--max-old-space-size=4096 NODE_ENV=production && webpack --bail --mode production. Also update the node version (last working version was v12.16.3).

  • Expected linebreaks to be 'LF' but found 'CRLF': Configure your IDE to use LF instead of CRLF. If that doesn't fix it, add "linebreak-style": 0 in "rules" in .estlintrc

  • ERROR: for electricitymap-contrib_mockserver_1 Cannot start service mockserver: OCI runtime create failed: container_linux.go:345: starting container process caused "process_linux.go:424: container init caused \"rootfs_linux.go:58: mounting \\\".../server.py\\\" to rootfs \\\"/mnt/sda1/var/lib/docker/overlay2/.../merged\\\" at \\\".../server.py\\\" caused \\\"not a directory\\\"\"": unknown: Are you trying to mount a directory onto a file (or vice-versa)? Check if the specified host path exists and is the expected type.

    • Check that the project is cloned under C:/Users/

  • nodemon not restarting on file changes: try adding the -L parameter to use the legacy watch: "server-dev": "nodemon server.js -L". See https://www.npmjs.com/package/nodemon#application-isnt-restarting

About

A real-time visualisation of the CO2 emissions of electricity consumption

www.electricitymap.org

Resources

Readme

License

MIT license
Activity

Stars

0 stars

Watchers

1 watching

Forks

0 forks
Report repository

Releases

201 tags

Packages

No packages published

Languages

  • Python 58.3%
  • JavaScript 26.5%
  • HTML 10.9%
  • CSS 3.5%
  • Jupyter Notebook 0.4%
  • Shell 0.3%
  • Dockerfile 0.1%