Releases: singularity-energy/open-grid-emissions
v0.1.2
Release Notes
Fixes issue with assignment of subplant IDs
Fixes an issue that was causing some generators/units to be assigned missing or incorrect subplant IDs. This issue caused several downstream issues including inaccurate hourly shapes being assigned to certain subplants, or inaccurate conversion of gross to net generation in the CEMS data. This patch ensures that every generator_id
and unitid
is associated with a non-missing subplant ID, and that these subplant assignments account for boiler-generator associations from EIA-860 (full details).
- Impacted data: Plant-level data, power sector data, and carbon accounting data
Fixes anomalous spikes in emission rate data
Several grid regions were exhibiting anomalous dips in their regional emission intensity values due to an issue with the methodology used to shape data from plants with partial CEMS data. Specifically, the generation from certain non-emitting plants (e.g. nuclear, solar, etc) that had a fossil-fuel backup generator onsite were being assigned the intermittent hourly profile of the backup generator if that generator reported to CEMS. This resulted in data quality issues in both the generated and consumed emission rates for some regions. This patch fixes that issue by excluding all non-emitting generators and plants with subplants of mixed fuel types from using the partial CEMS methodology. (full details).
- Impacted data: carbon accounting data, power sector data, certain plant data, with the largest impacts in PJM and plant EIA ID = 2410 (PSEG Salem Generation Station)
Other updates
- Fixes an issue that was resulting in an infeasible conda environment by updating to a stable branch of the Public Utility Data Liberation Project.
- Improves the speed of running the part of the data pipeline that identifies subplant IDs.
- Updates the
plant_metadata.csv
file to help users more easily identify the methodologies used for each plant. - Adds adjusted R-squared values to the gross to net generation regression outputs available in the
data/outputs/gross_to_net_conversions.csv
file. - Renames the
data/outputs/subplant_crosswalk.csv
file tosubplant_crosswalk_[YEAR].csv
to clarify that subplant IDs are only valid for a specific year.
What's Changed
- update pudl dependency by @grgmiller in #235
- Update subplant mapping by @grgmiller in #239
- Validate that data does not overlap when combining by @grgmiller in #245
- Update partial CEMS shaping for mixed fuel plants by @gailin-p in #238
- Add BA-level data quality metrics by @grgmiller in #233
- v0.1.2 by @grgmiller in #251
Full Changelog: v0.1.1...v0.1.2
v0.1.1
Summary of updates
This release (v0.1.1) patches several bugs that were affecting the quality of the data published in the initial release. A big thank you to user @ewezerek for bringing one of these bugs to our attention.
Consumed emission factors
Due to a mathematical error in the previous version of the code, the consumed emissions factors incorrectly accounted for the flow of electricity between regions, which was causing some consumed emissions factors to be reported as negative. We have fixed this error and also implemented validation checks to ensure that the published emissions factors are not negative. While fixing this bug, we also noticed a data quality issue with the EIA-930 input data for AZPS interchange with SRP prior to 6/1/2020, which has been manually corrected.
Impacted data: all carbon accounting data, certain regional power sector data
Sulfur Dioxide (SO2) emissions
Due to misunderstanding several of the assumptions for applying SO2 emissions factors, the SO2 emissions data derived from EIA-923 fuel consumption data were under-reported by several orders of magnitude. These calculations have been corrected, and the updated SO2 emissions totals have been validated as consistent with the SO2 emissions totals published in other data sources.
Impacted data: All SO2 emissions data for plants that don’t report SO2 emissions to CEMS (impacts all regional power sector and emissions accounting data)
Data Quality Metrics
This release adds NOx and SO2 data to the published data quality metrics, and adds information about the quality of the published CEMS data. Details on these updates can be found here.
What's Changed
- Fix bugs with SO2 calculations by @grgmiller in #219
- Update data quality metric outputs by @grgmiller in #217
- Gailin/fix consumed sign bug by @gailin-p in #221
- Gailin/azps patch by @gailin-p in #222
- Version 0.1.1 by @grgmiller in #223
Full Changelog: v0.1.0...v0.1.1
v0.1.0
What's Changed
- Simple script to download data sources by @gailin-p in #1
- Gailinpease/car 96 cems only estimate by @gailin-p in #2
- Gailinpease/car 115 eia 930 data access by @gailin-p in #3
- Grmiller/car 85 create data pipeline by @grgmiller in #4
- Gailinpease/car 116 compare aggregated generation by @gailin-p in #5
- Add cems coverage notebook by @gailin-p in #7
- Grmiller/car 216 classify plants by data coverage by @grgmiller in #9
- Grmiller/car 124 clean eia 923 net generation and heat by @grgmiller in #10
- Added new table C5 by @gailin-p in #11
- Gailinpease/car 118 cleaning of eia 930 by @gailin-p in #12
- Grmiller/car 124 clean eia 923 net generation and heat by @grgmiller in #13
- CEMS-EIA930 Matching by @grgmiller in #14
- Gailinpease/car 271 evaluate lag in 930 data by @gailin-p in #15
- Grmiller/car 124 clean eia 923 net generation and heat by @grgmiller in #16
- Calculate NOx, SO2, and CO2e from EIA-923 by @miloknowles in #21
- Grmiller/car 342 clean cems and eia923 by @grgmiller in #24
- Move data_pipeline to script by @gailin-p in #29
- Add function to compute CO2-eq emissions by @miloknowles in #25
- partial_cems calculations by @grgmiller in #36
- Hourly profiles for clean gens by @grgmiller in #38
- fix primary fuel mode by @grgmiller in #40
- Gailinpease/car 363 compare residual profile to cems plants by @gailin-p in #39
- update results and organize code by @grgmiller in #43
- remove CHANGELOG.md and migrate ISSUES.md to Github issues by @grgmiller in #65
- Fix missing output data by @grgmiller in #84
- Gailin/consumption by @gailin-p in #88
- Greg/memory overflow by @grgmiller in #89
- Aggregate EIA data to BA before hourly shaping by @gailin-p in #91
- Fix BA coverage and missing BA codes by @grgmiller in #92
- fix profile imputation by @grgmiller in #98
- Gailin/output formatting by @gailin-p in #95
- Add artificial BA/fuel plants to plant frame by @gailin-p in #101
- Plant-level outputs by @gailin-p in #106
- fix gross to net method by @grgmiller in #107
- Integrate 930 into pipeline by @gailin-p in #105
- fix primary fuel and Nox adjustment by @grgmiller in #109
- Revert "fix primary fuel and Nox adjustment" by @gailin-p in #116
- Gailin/streamline 930 by @gailin-p in #115
- Greg/fix residual scaling by @grgmiller in #112
- fix missing fuel and BA codes by @grgmiller in #119
- patch eia930 imputation issues by @grgmiller in #123
- update method for hourly shaping by @grgmiller in #126
- Gailin/fix 2019 cems by @gailin-p in #127
- Validate gtn by @grgmiller in #129
- Validation Fixes by @grgmiller in #131
- update egrid validation notebook by @grgmiller in #132
- Fix CHP calculations by @grgmiller in #133
- small fixes for 2019 data by @grgmiller in #135
- fix missing subplant ids by @grgmiller in #136
- update to python 3.10 by @grgmiller in #138
- Gailin/update cems residual requirement by @gailin-p in #140
- hourly profile updates by @grgmiller in #142
- add validation metrics by @grgmiller in #144
- fix dropped data by @grgmiller in #149
- Update NOx and SO2 calculations by @grgmiller in #152
- Update Gross to Net Generation Hierarchy by @grgmiller in #157
- Update egrid notebook by @grgmiller in #160
- Gailin/fix consumed by @gailin-p in #162
- Clean up readme and files by @grgmiller in #165
- Gailin/validate carbon by @gailin-p in #145
- fix merge validation errors by @grgmiller in #173
- Update
clean_eia923
to work with pre-2019 data by @grgmiller in #179 - investigate annually-reported data by @grgmiller in #180
- Add tests to ensure that data download and EIA-923 cleaning works for 2005-2020 by @miloknowles in #181
- Update egrid validation by @grgmiller in #182
- Fix incorrect column name in load_diba_data function by @miloknowles in #183
- Shape partial plants by @grgmiller in #184
- Fix mismatched timezone futurewarning by @grgmiller in #189
- Visualization updates and bug fixes by @grgmiller in #196
- Gailin/map viz by @gailin-p in #192
- Use updated datasette by @grgmiller in #197
- fix missing geothermal nox emissons by @grgmiller in #198
- Fix bug where consumed.py did not have enough 930 hours at end of year by @gailin-p in #199
- Fix co2e by @grgmiller in #200
- fix monthly totals by @grgmiller in #201
- fix duplicate power sector datetitmes by @grgmiller in #202
- Add notebook for zipping files by @grgmiller in #204
- Gailin/map viz by @gailin-p in #205
- Create LICENSE.md by @grgmiller in #206
- add function for rounding outputs by @gailin-p in #207
- delete .vscode by @grgmiller in #208
- update gitignore by @grgmiller in #209
New Contributors
- @gailin-p made their first contribution in #1
- @grgmiller made their first contribution in #4
- @miloknowles m...