Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Changes on generic_metric and mode_specific_metrics in relation to Mo… #91

Merged

Conversation

iantei
Copy link
Contributor

@iantei iantei commented Sep 7, 2023

…de_confirm mapping with dynamic labels.

@iantei iantei marked this pull request as ready for review September 7, 2023 16:04
@iantei iantei marked this pull request as draft September 7, 2023 16:05
@iantei iantei marked this pull request as ready for review September 7, 2023 16:55
Copy link
Contributor

@shankari shankari left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

please indicate testing done as well.
How do you know that this works?

viz_scripts/bin/generate_plots.py Outdated Show resolved Hide resolved
viz_scripts/bin/generate_plots.py Outdated Show resolved Hide resolved
viz_scripts/generic_metrics.ipynb Outdated Show resolved Hide resolved
viz_scripts/scaffolding.py Outdated Show resolved Hide resolved
viz_scripts/scaffolding.py Outdated Show resolved Hide resolved
viz_scripts/scaffolding.py Outdated Show resolved Hide resolved
@iantei
Copy link
Contributor Author

iantei commented Sep 11, 2023

please indicate testing done as well.
How do you know that this works?

I have executed manual testing, by modifying the year and month parameter to run for different notebooks.
Below:

Mode confirmation for Trips under 10 miles Number of Trips for each mode
Execution for generic_metrics with params: Year: None, Month: None, dynamic_labels loaded with json object from dynamic_labels Similarly, execution for generic_metrics notebook with params changed to: Year: 2020, Month: 11, dynamic_labels as empty
generic_metrics generic_metric_empty_dynamic_labels

Copy link
Contributor

@shankari shankari left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

almost there. the requested changes are fairly minor.
Please remember to resolve the merge conflict.

We need the most improvement in the "Testing done"

Your existing testing:

  • does not touch generate_plots.py
  • does not actually test the new custom label functionality (I don't see any entries for the one new mode = 'Moped')
  • does not compare the same metric with and without dynamic labels to illustrate the difference with and without using custom labels

Code review is not a substitute for testing. It focuses on readability and code structure. We still need to actually test to make sure that the code works (aka "if it is not tested, it is broken" 😄 )

viz_scripts/bin/generate_plots.py Outdated Show resolved Hide resolved
viz_scripts/bin/generate_plots.py Outdated Show resolved Hide resolved
viz_scripts/energy_calculations.ipynb Outdated Show resolved Hide resolved
viz_scripts/generic_timeseries.ipynb Outdated Show resolved Hide resolved
viz_scripts/mode_specific_metrics.ipynb Outdated Show resolved Hide resolved
viz_scripts/mode_specific_timeseries.ipynb Outdated Show resolved Hide resolved
viz_scripts/scaffolding.py Outdated Show resolved Hide resolved
viz_scripts/scaffolding.py Outdated Show resolved Hide resolved
@iantei
Copy link
Contributor Author

iantei commented Sep 12, 2023

Some testing cases:

  1. In case of energy_calculations notebook:
    In both these cases, the Month and Year have been set to "None".
With Default Mapping With Dynamic Labels
Mapping with auxillary file Mapping with dynamic label json
dynamic_labels is passed as { } dynamic_labels is passed as json object available on Dynamic labels json object example

The difference which is apparent, is the difference in mapping label for Replaced Mode on Y-axis - "Gas Car, Others" vs "Gas Car Shared Ride".

@iantei
Copy link
Contributor Author

iantei commented Sep 14, 2023

Testing scenario:

Distribution of Mode_confirm attribute without dynamic_labels: Distribution of Mode_confirm attribute without dynamic_labels

Distribution of Mode_confirm attribute with dynamic_labels:
Added 6 entries into the database, containing moped as user_input.mode_confirm.
The entry for mode_confirm is less than 2%, therefore is merged into Others category in the pie chart representation.
It is represented as "Moped" in the table below.
Moped_6entries

Another representation of "Moped" label, showing dynamic_labels:
Average miles per transport mode selected:
Average_miles_per_transport_mode_selected(Mode_confirm)

Ran notebook: generic metrics notebook through automation

  1. Passing dynamic_labels json object through generate_plots.py:
Notebook_generic_metrics_automation_run
  1. Without passing dynamic_labels json object through generate_plots.py, initializing it as dynamic_labels = { }
Notebook_automation_empty_dynamic_labels

I will experiment with more entries of Moped data, if required.

@iantei
Copy link
Contributor Author

iantei commented Sep 14, 2023

almost there. the requested changes are fairly minor. Please remember to resolve the merge conflict.

We need the most improvement in the "Testing done"

Your existing testing:

  • does not touch generate_plots.py
  • does not actually test the new custom label functionality (I don't see any entries for the one new mode = 'Moped')
  • does not compare the same metric with and without dynamic labels to illustrate the difference with and without using custom labels

Code review is not a substitute for testing. It focuses on readability and code structure. We still need to actually test to make sure that the code works (aka "if it is not tested, it is broken" 😄 )

Executed all these testing, mentioned above.

@shankari
Copy link
Contributor

@iantei

Added 6 entries into the database, containing moped as user_input.mode_confirm.
The entry for mode_confirm is less than 2%, therefore is merged into Others category in the pie chart representation.
It is represented as "Moped" in the table below.

High level feedback: instead of a one-time update saying "I did X", it would be good to record what you did in this "lab notebook" - aka I did X in this way. As you can see from my PR, that helps maintain a record of exactly what was done for reproducibility, and also helps others who want to achieve the same goal.

Ran notebook: generic metrics notebook through automation
Passing dynamic_labels json object through generate_plots.py:

High level feedback: Please don't put code into issues in screenshots. Screenshots are not searchable, or copy-pasteable or zoomable... Please replace with markdown code blocks.

I will experiment with more entries of Moped data, if required.

Yes, you should have more entries of Moped data - neither of your tests shows any actual data. I don't understand why you didn't just replace all entries of a mode that is present - e.g. walk -> moped and then check that the graph was the same, only with the moped label instead of walk

Passing dynamic_labels json object through generate_plots.py:
Without passing dynamic_labels json object through generate_plots.py, initializing it as dynamic_labels = { }

Both of these are calling the same script in the same way with the same config file. Running tests by editing the code between tests is not principled - you are not testing the code that is going to be committed. Please run the code with different configurations to verify.

@iantei
Copy link
Contributor Author

iantei commented Sep 17, 2023

**Updated Testing scenario:**

Testing done:
Notebook runs through automation:

With the default settings: STUDY_CONFIG:stage-program

Results are: generic_metrics.ipynb:

(emission) root@af656e62a649:/usr/src/app/saved-notebooks# PYTHONPATH=.. python bin/update_mappings.py mapping_dictionaries.ipynb
(emission) root@af656e62a649:/usr/src/app/saved-notebooks# PYTHONPATH=.. python bin/generate_plots.py generic_metrics.ipynb
/usr/src/app/saved-notebooks/bin/generate_plots.py:30: SyntaxWarning: "is not" with a literal. Did you mean "!="?
  if r.status_code is not 200:
usage: generate_metrics [-h] [-d DATE DATE] plot_notebook program
generate_metrics: error: the following arguments are required: program
(emission) root@af656e62a649:/usr/src/app/saved-notebooks# PYTHONPATH=.. python bin/generate_plots.py generic_metrics.ipynb default
/usr/src/app/saved-notebooks/bin/generate_plots.py:30: SyntaxWarning: "is not" with a literal. Did you mean "!="?
  if r.status_code is not 200:
About to download config from https://raw.githubusercontent.com/e-mission/nrel-openpath-deploy-configs/main/configs/stage-program.nrel-op.json
Successfully downloaded config with version 1 for Staging environment for testing programs only and data collection URL https://openpath-stage.nrel.gov/api/
Dynamic labels are not available.
Running at 2023-09-19T17:54:27.240880+00:00 with args Namespace(plot_notebook='generic_metrics.ipynb', program='default', date=None) for range (<Arrow [2020-09-01T00:00:00+00:00]>, <Arrow [2023-09-01T00:00:00+00:00]>)
Running at 2023-09-19T17:54:27.279311+00:00 with params [Parameter('year', int), Parameter('month', int), Parameter('program', str, value='default'), Parameter('study_type', str, value='program'), Parameter('include_test_users', bool, value=True), Parameter('dynamic_labels', dict, value={})]
Running at 2023-09-19T17:54:52.317446+00:00 with params [Parameter('year', int, value=2020), Parameter('month', int, value=9), Parameter('program', str, value='default'), Parameter('study_type', str, value='program'), Parameter('include_test_users', bool, value=True), Parameter('dynamic_labels', dict, value={})]
Running at 2023-09-19T17:54:59.014789+00:00 with params [Parameter('year', int, value=2020), Parameter('month', int, value=10), Parameter('program', str, value='default'), Parameter('study_type', str, value='program'), Parameter('include_test_users', bool, value=True), Parameter('dynamic_labels', dict, value={})]
Running at 2023-09-19T17:55:05.722502+00:00 with params [Parameter('year', int, value=2020), Parameter('month', int, value=11), Parameter('program', str, value='default'), Parameter('study_type', str, value='program'), Parameter('include_test_users', bool, value=True), Parameter('dynamic_labels', dict, value={})]
Running at 2023-09-19T17:55:13.210319+00:00 with params [Parameter('year', int, value=2020), Parameter('month', int, value=12), Parameter('program', str, value='default'), Parameter('study_type', str, value='program'), Parameter('include_test_users', bool, value=True), Parameter('dynamic_labels', dict, value={})]

With the change: STUDY_CONFIG:dev-emulator-study

ashrest2-35384s:em-public-dashboard ashrest2$ git diff docker-compose.dev.yml 
diff --git a/docker-compose.dev.yml b/docker-compose.dev.yml
index fdbdbb8..9cbc4de 100644
--- a/docker-compose.dev.yml
+++ b/docker-compose.dev.yml
@@ -26,7 +26,7 @@ services:
       - DB_HOST=db
       - WEB_SERVER_HOST=0.0.0.0
       - CRON_MODE=
-      - STUDY_CONFIG=stage-program
+      - STUDY_CONFIG=dev-emulator-study
     ports:
       # ipynb in numbers
       - "47962:8888"

Results are: generic_metrics.ipynb

(emission) root@f9305d3a5550:/usr/src/app/saved-notebooks# PYTHONPATH=.. python bin/update_mappings.py mapping_dictionaries.ipynb
(emission) root@f9305d3a5550:/usr/src/app/saved-notebooks# PYTHONPATH=.. python bin/generate_plots.py generic_metrics.ipynb default
/usr/src/app/saved-notebooks/bin/generate_plots.py:30: SyntaxWarning: "is not" with a literal. Did you mean "!="?
  if r.status_code is not 200:
About to download config from https://raw.githubusercontent.com/e-mission/nrel-openpath-deploy-configs/main/configs/dev-emulator-study.nrel-op.json
Successfully downloaded config with version 1 for Development environment (study) and data collection URL default
Dynamic labels download was successful.
Running at 2023-09-19T18:17:43.139687+00:00 with args Namespace(plot_notebook='generic_metrics.ipynb', program='default', date=None) for range (<Arrow [2020-09-01T00:00:00+00:00]>, <Arrow [2023-09-01T00:00:00+00:00]>)
Running at 2023-09-19T18:17:43.178087+00:00 with params [Parameter('year', int), Parameter('month', int), Parameter('program', str, value='default'), Parameter('study_type', str, value='study'), Parameter('include_test_users', bool, value=False), Parameter('dynamic_labels', dict, value={'MODE': [{'value': 'walk', 'baseMode': 'WALKING', 'met_equivalent': 'WALKING', 'kgCo2PerKm': 0}, {'value': 'e-bike', 'baseMode': 'E_BIKE', 'met': {'ALL': {'range': [0, -1], 'mets': 4.9}}, 'kgCo2PerKm': 0.00728}, {'value': 'bike', 'baseMode': 'BICYCLING', 'met_equivalent': 'BICYCLING', 'kgCo2PerKm': 0}, {'value': 'bikeshare', 'baseMode': 'BICYCLING', 'met_equivalent': 'BICYCLING', 'kgCo2PerKm': 0}, {'value': 'scootershare', 'baseMode': 'E_SCOOTER', 'met_equivalent': 'IN_VEHICLE', 'kgCo2PerKm': 0.00894}, {'value': 'drove_alone', 'baseMode': 'CAR', 'met_equivalent': 'IN_VEHICLE', 'kgCo2PerKm': 0.22031}, {'value': 'shared_ride', 'baseMode': 'CAR', 'met_equivalent': 'IN_VEHICLE', 'kgCo2PerKm': 0.11015}, {'value': 'e_car_drove_alone', 'baseMode': 'E_CAR', 'met_equivalent': 'IN_VEHICLE', 'kgCo2PerKm': 0.08216}, {'value': 'e_car_shared_ride', 'baseMode': 'E_CAR', 'met_equivalent': 'IN_VEHICLE', 'kgCo2PerKm': 0.04108}, {'value': 'moped', 'baseMode': 'MOPED', 'met_equivalent': 'IN_VEHICLE', 'kgCo2PerKm': 0.05555}, {'value': 'taxi', 'baseMode': 'TAXI', 'met_equivalent': 'IN_VEHICLE', 'kgCo2PerKm': 0.30741}, {'value': 'bus', 'baseMode': 'BUS', 'met_equivalent': 'IN_VEHICLE', 'kgCo2PerKm': 0.20727}, {'value': 'train', 'baseMode': 'TRAIN', 'met_equivalent': 'IN_VEHICLE', 'kgCo2PerKm': 0.12256}, {'value': 'free_shuttle', 'baseMode': 'BUS', 'met_equivalent': 'IN_VEHICLE', 'kgCo2PerKm': 0.20727}, {'value': 'air', 'baseMode': 'AIR', 'met_equivalent': 'IN_VEHICLE', 'kgCo2PerKm': 0.09975}, {'value': 'not_a_trip', 'baseMode': 'UNKNOWN', 'met_equivalent': 'UNKNOWN', 'kgCo2PerKm': 0}, {'value': 'other', 'baseMode': 'OTHER', 'met_equivalent': 'UNKNOWN', 'kgCo2PerKm': 0}], 'PURPOSE': [{'value': 'home'}, {'value': 'work'}, {'value': 'at_work'}, {'value': 'school'}, {'value': 'transit_transfer'}, {'value': 'shopping'}, {'value': 'meal'}, {'value': 'pick_drop_person'}, {'value': 'pick_drop_item'}, {'value': 'personal_med'}, {'value': 'access_recreation'}, {'value': 'exercise'}, {'value': 'entertainment'}, {'value': 'religious'}, {'value': 'other'}], 'translations': {'en': {'walk': 'Walk', 'e-bike': 'E-bike', 'bike': 'Regular Bike', 'bikeshare': 'Bikeshare', 'scootershare': 'Scooter share', 'drove_alone': 'Gas Car Drove Alone', 'shared_ride': 'Gas Car Shared Ride', 'e_car_drove_alone': 'E-Car Drove Alone', 'e_car_shared_ride': 'E-Car Shared Ride', 'moped': 'Moped', 'taxi': 'Taxi/Uber/Lyft', 'bus': 'Bus', 'train': 'Train', 'free_shuttle': 'Free Shuttle', 'air': 'Air', 'not_a_trip': 'Not a trip', 'home': 'Home', 'work': 'To Work', 'at_work': 'At Work', 'school': 'School', 'transit_transfer': 'Transit transfer', 'shopping': 'Shopping', 'meal': 'Meal', 'pick_drop_person': 'Pick-up/ Drop off Person', 'pick_drop_item': 'Pick-up/ Drop off Item', 'personal_med': 'Personal/ Medical', 'access_recreation': 'Access Recreation', 'exercise': 'Recreation/ Exercise', 'entertainment': 'Entertainment/ Social', 'religious': 'Religious', 'other': 'Other'}, 'es': {'walk': 'Caminando', 'e-bike': 'e-bicicleta', 'bike': 'Bicicleta', 'bikeshare': 'Bicicleta compartida', 'scootershare': 'Motoneta compartida', 'drove_alone': 'Coche de Gas, Condujo solo', 'shared_ride': 'Coche de Gas, Condujo con otros', 'e_car_drove_alone': 'e-coche, Condujo solo', 'e_car_shared_ride': 'e-coche, Condujo con ontras', 'moped': 'Ciclomotor', 'taxi': 'Taxi/Uber/Lyft', 'bus': 'Autobús', 'train': 'Tren', 'free_shuttle': 'Colectivo gratuito', 'air': 'Avión', 'not_a_trip': 'No es un viaje', 'home': 'Inicio', 'work': 'Trabajo', 'at_work': 'En el trabajo', 'school': 'Escuela', 'transit_transfer': 'Transbordo', 'shopping': 'Compras', 'meal': 'Comida', 'pick_drop_person': 'Recoger/ Entregar Individuo', 'pick_drop_item': 'Recoger/ Entregar Objeto', 'personal_med': 'Personal/ Médico', 'access_recreation': 'Acceder a Recreación', 'exercise': 'Recreación/ Ejercicio', 'entertainment': 'Entretenimiento/ Social', 'religious': 'Religioso', 'other': 'Otros'}}})]
Running at 2023-09-19T18:18:02.744057+00:00 with params [Parameter('year', int, value=2020), Parameter('month', int, value=9), Parameter('program', str, value='default'), Parameter('study_type', str, value='study'), Parameter('include_test_users', bool, value=False), Parameter('dynamic_labels', dict, value={'MODE': [{'value': 'walk', 'baseMode': 'WALKING', 'met_equivalent': 'WALKING', 'kgCo2PerKm': 0}, {'value': 'e-bike', 'baseMode': 'E_BIKE', 'met': {'ALL': {'range': [0, -1], 'mets': 4.9}}, 'kgCo2PerKm': 0.00728}, {'value': 'bike', 'baseMode': 'BICYCLING', 'met_equivalent': 'BICYCLING', 'kgCo2PerKm': 0}, {'value': 'bikeshare', 'baseMode': 'BICYCLING', 'met_equivalent': 'BICYCLING', 'kgCo2PerKm': 0}, {'value': 'scootershare', 'baseMode': 'E_SCOOTER', 'met_equivalent': 'IN_VEHICLE', 'kgCo2PerKm': 0.00894}, {'value': 'drove_alone', 'baseMode': 'CAR', 'met_equivalent': 'IN_VEHICLE', 'kgCo2PerKm': 0.22031}, {'value': 'shared_ride', 'baseMode': 'CAR', 'met_equivalent': 'IN_VEHICLE', 'kgCo2PerKm': 0.11015}, {'value': 'e_car_drove_alone', 'baseMode': 'E_CAR', 'met_equivalent': 'IN_VEHICLE', 'kgCo2PerKm': 0.08216}, {'value': 'e_car_shared_ride', 'baseMode': 'E_CAR', 'met_equivalent': 'IN_VEHICLE', 'kgCo2PerKm': 0.04108}, {'value': 'moped', 'baseMode': 'MOPED', 'met_equivalent': 'IN_VEHICLE', 'kgCo2PerKm': 0.05555}, {'value': 'taxi', 'baseMode': 'TAXI', 'met_equivalent': 'IN_VEHICLE', 'kgCo2PerKm': 0.30741}, {'value': 'bus', 'baseMode': 'BUS', 'met_equivalent': 'IN_VEHICLE', 'kgCo2PerKm': 0.20727}, {'value': 'train', 'baseMode': 'TRAIN', 'met_equivalent': 'IN_VEHICLE', 'kgCo2PerKm': 0.12256}, {'value': 'free_shuttle', 'baseMode': 'BUS', 'met_equivalent': 'IN_VEHICLE', 'kgCo2PerKm': 0.20727}, {'value': 'air', 'baseMode': 'AIR', 'met_equivalent': 'IN_VEHICLE', 'kgCo2PerKm': 0.09975}, {'value': 'not_a_trip', 'baseMode': 'UNKNOWN', 'met_equivalent': 'UNKNOWN', 'kgCo2PerKm': 0}, {'value': 'other', 'baseMode': 'OTHER', 'met_equivalent': 'UNKNOWN', 'kgCo2PerKm': 0}], 'PURPOSE': [{'value': 'home'}, {'value': 'work'}, {'value': 'at_work'}, {'value': 'school'}, {'value': 'transit_transfer'}, {'value': 'shopping'}, {'value': 'meal'}, {'value': 'pick_drop_person'}, {'value': 'pick_drop_item'}, {'value': 'personal_med'}, {'value': 'access_recreation'}, {'value': 'exercise'}, {'value': 'entertainment'}, {'value': 'religious'}, {'value': 'other'}], 'translations': {'en': {'walk': 'Walk', 'e-bike': 'E-bike', 'bike': 'Regular Bike', 'bikeshare': 'Bikeshare', 'scootershare': 'Scooter share', 'drove_alone': 'Gas Car Drove Alone', 'shared_ride': 'Gas Car Shared Ride', 'e_car_drove_alone': 'E-Car Drove Alone', 'e_car_shared_ride': 'E-Car Shared Ride', 'moped': 'Moped', 'taxi': 'Taxi/Uber/Lyft', 'bus': 'Bus', 'train': 'Train', 'free_shuttle': 'Free Shuttle', 'air': 'Air', 'not_a_trip': 'Not a trip', 'home': 'Home', 'work': 'To Work', 'at_work': 'At Work', 'school': 'School', 'transit_transfer': 'Transit transfer', 'shopping': 'Shopping', 'meal': 'Meal', 'pick_drop_person': 'Pick-up/ Drop off Person', 'pick_drop_item': 'Pick-up/ Drop off Item', 'personal_med': 'Personal/ Medical', 'access_recreation': 'Access Recreation', 'exercise': 'Recreation/ Exercise', 'entertainment': 'Entertainment/ Social', 'religious': 'Religious', 'other': 'Other'}, 'es': {'walk': 'Caminando', 'e-bike': 'e-bicicleta', 'bike': 'Bicicleta', 'bikeshare': 'Bicicleta compartida', 'scootershare': 'Motoneta compartida', 'drove_alone': 'Coche de Gas, Condujo solo', 'shared_ride': 'Coche de Gas, Condujo con otros', 'e_car_drove_alone': 'e-coche, Condujo solo', 'e_car_shared_ride': 'e-coche, Condujo con ontras', 'moped': 'Ciclomotor', 'taxi': 'Taxi/Uber/Lyft', 'bus': 'Autobús', 'train': 'Tren', 'free_shuttle': 'Colectivo gratuito', 'air': 'Avión', 'not_a_trip': 'No es un viaje', 'home': 'Inicio', 'work': 'Trabajo', 'at_work': 'En el trabajo', 'school': 'Escuela', 'transit_transfer': 'Transbordo', 'shopping': 'Compras', 'meal': 'Comida', 'pick_drop_person': 'Recoger/ Entregar Individuo', 'pick_drop_item': 'Recoger/ Entregar Objeto', 'personal_med': 'Personal/ Médico', 'access_recreation': 'Acceder a Recreación', 'exercise': 'Recreación/ Ejercicio', 'entertainment': 'Entretenimiento/ Social', 'religious': 'Religioso', 'other': 'Otros'}}})]
Running at 2023-09-19T18:18:09.989461+00:00 with params [Parameter('year', int, value=2020), Parameter('month', int, value=10), Parameter('program', str, value='default'), Parameter('study_type', str, value='study'), Parameter('include_test_users', bool, value=False), Parameter('dynamic_labels', dict, value={'MODE': [{'value': 'walk', 'baseMode': 'WALKING', 'met_equivalent': 'WALKING', 'kgCo2PerKm': 0}, {'value': 'e-bike', 'baseMode': 'E_BIKE', 'met': {'ALL': {'range': [0, -1], 'mets': 4.9}}, 'kgCo2PerKm': 0.00728}, {'value': 'bike', 'baseMode': 'BICYCLING', 'met_equivalent': 'BICYCLING', 'kgCo2PerKm': 0}, {'value': 'bikeshare', 'baseMode': 'BICYCLING', 'met_equivalent': 'BICYCLING', 'kgCo2PerKm': 0}, {'value': 'scootershare', 'baseMode': 'E_SCOOTER', 'met_equivalent': 'IN_VEHICLE', 'kgCo2PerKm': 0.00894}, {'value': 'drove_alone', 'baseMode': 'CAR', 'met_equivalent': 'IN_VEHICLE', 'kgCo2PerKm': 0.22031}, {'value': 'shared_ride', 'baseMode': 'CAR', 'met_equivalent': 'IN_VEHICLE', 'kgCo2PerKm': 0.11015}, {'value': 'e_car_drove_alone', 'baseMode': 'E_CAR', 'met_equivalent': 'IN_VEHICLE', 'kgCo2PerKm': 0.08216}, {'value': 'e_car_shared_ride', 'baseMode': 'E_CAR', 'met_equivalent': 'IN_VEHICLE', 'kgCo2PerKm': 0.04108}, {'value': 'moped', 'baseMode': 'MOPED', 'met_equivalent': 'IN_VEHICLE', 'kgCo2PerKm': 0.05555}, {'value': 'taxi', 'baseMode': 'TAXI', 'met_equivalent': 'IN_VEHICLE', 'kgCo2PerKm': 0.30741}, {'value': 'bus', 'baseMode': 'BUS', 'met_equivalent': 'IN_VEHICLE', 'kgCo2PerKm': 0.20727}, {'value': 'train', 'baseMode': 'TRAIN', 'met_equivalent': 'IN_VEHICLE', 'kgCo2PerKm': 0.12256}, {'value': 'free_shuttle', 'baseMode': 'BUS', 'met_equivalent': 'IN_VEHICLE', 'kgCo2PerKm': 0.20727}, {'value': 'air', 'baseMode': 'AIR', 'met_equivalent': 'IN_VEHICLE', 'kgCo2PerKm': 0.09975}, {'value': 'not_a_trip', 'baseMode': 'UNKNOWN', 'met_equivalent': 'UNKNOWN', 'kgCo2PerKm': 0}, {'value': 'other', 'baseMode': 'OTHER', 'met_equivalent': 'UNKNOWN', 'kgCo2PerKm': 0}], 'PURPOSE': [{'value': 'home'}, {'value': 'work'}, {'value': 'at_work'}, {'value': 'school'}, {'value': 'transit_transfer'}, {'value': 'shopping'}, {'value': 'meal'}, {'value': 'pick_drop_person'}, {'value': 'pick_drop_item'}, {'value': 'personal_med'}, {'value': 'access_recreation'}, {'value': 'exercise'}, {'value': 'entertainment'}, {'value': 'religious'}, {'value': 'other'}], 'translations': {'en': {'walk': 'Walk', 'e-bike': 'E-bike', 'bike': 'Regular Bike', 'bikeshare': 'Bikeshare', 'scootershare': 'Scooter share', 'drove_alone': 'Gas Car Drove Alone', 'shared_ride': 'Gas Car Shared Ride', 'e_car_drove_alone': 'E-Car Drove Alone', 'e_car_shared_ride': 'E-Car Shared Ride', 'moped': 'Moped', 'taxi': 'Taxi/Uber/Lyft', 'bus': 'Bus', 'train': 'Train', 'free_shuttle': 'Free Shuttle', 'air': 'Air', 'not_a_trip': 'Not a trip', 'home': 'Home', 'work': 'To Work', 'at_work': 'At Work', 'school': 'School', 'transit_transfer': 'Transit transfer', 'shopping': 'Shopping', 'meal': 'Meal', 'pick_drop_person': 'Pick-up/ Drop off Person', 'pick_drop_item': 'Pick-up/ Drop off Item', 'personal_med': 'Personal/ Medical', 'access_recreation': 'Access Recreation', 'exercise': 'Recreation/ Exercise', 'entertainment': 'Entertainment/ Social', 'religious': 'Religious', 'other': 'Other'}, 'es': {'walk': 'Caminando', 'e-bike': 'e-bicicleta', 'bike': 'Bicicleta', 'bikeshare': 'Bicicleta compartida', 'scootershare': 'Motoneta compartida', 'drove_alone': 'Coche de Gas, Condujo solo', 'shared_ride': 'Coche de Gas, Condujo con otros', 'e_car_drove_alone': 'e-coche, Condujo solo', 'e_car_shared_ride': 'e-coche, Condujo con ontras', 'moped': 'Ciclomotor', 'taxi': 'Taxi/Uber/Lyft', 'bus': 'Autobús', 'train': 'Tren', 'free_shuttle': 'Colectivo gratuito', 'air': 'Avión', 'not_a_trip': 'No es un viaje', 'home': 'Inicio', 'work': 'Trabajo', 'at_work': 'En el trabajo', 'school': 'Escuela', 'transit_transfer': 'Transbordo', 'shopping': 'Compras', 'meal': 'Comida', 'pick_drop_person': 'Recoger/ Entregar Individuo', 'pick_drop_item': 'Recoger/ Entregar Objeto', 'personal_med': 'Personal/ Médico', 'access_recreation': 'Acceder a Recreación', 'exercise': 'Recreación/ Ejercicio', 'entertainment': 'Entretenimiento/ Social', 'religious': 'Religioso', 'other': 'Otros'}}})]

Test Done Manually:
Before:
Looked up in the mongo container for the number of entries for "walk"

> show dbs

> use Stage_database
 switched to db Stage_database

> show collections
 CarbonUsage
 Checkinout
 New_tier
 PolarBear
 Stage_Profiles
 Stage_analysis_timeseries
 Stage_pipeline_state
 Stage_timeseries
 Stage_timeseries_error
 Stage_usercache
 Stage_uuids
 Tier_Sys
 Usernames

> db.Stage_analysis_timeseries.find({"data.user_input.mode_confirm":"walk"}).count()
4359

> db.Stage_analysis_timeseries.find({"data.user_input.mode_confirm":"moped"}).count()
0
Generic_Metrics_Before

Next:
Executed following code inside the mongo container to modify the "data.user_input.mode_confirm":"walk" to "data.user_input.mode_confirm":"moped"

var result_walk = db.Stage_analysis_timeseries.find({"data.user_input.mode_confirm":"walk"}, {"_id": 1})

var resultArray = result_walk.toArray();
for (var i = 0; i < resultArray.length; i++) {     var doc = resultArray[i];     var docId = doc._id;     var updateResult = db.Stage_analysis_timeseries.update(         { "_id": docId, "data.user_input.mode_confirm": "walk" },         { "$set": { "data.user_input.mode_confirm": "moped" } }     );     if (updateResult.nModified > 0) {         print(`Updated document with _id: ${docId}`);     } }

> db.Stage_analysis_timeseries.find({"data.user_input.mode_confirm":"moped"}).count()
4359
> db.Stage_analysis_timeseries.find({"data.user_input.mode_confirm":"walk"}).count()
0

Current Representation
Notebook: generic_metrics: With empty dynamic_labels:
There is no representation of walk - since it's not available, and moped is not mapped correctly, so it's categorized as Other.
Without_Dynamic_Config

Next:
Passing the following parameter into the notebook for dynamic_labels: dev-emulator-study label_options

{
  "MODE": [
    {"value":"walk", "baseMode":"WALKING", "met_equivalent":"WALKING", "kgCo2PerKm": 0},
    {"value":"e-bike", "baseMode":"E_BIKE", "met": {"ALL": {"range": [0, -1], "mets": 4.9}}, "kgCo2PerKm": 0.00728},
    {"value":"bike", "baseMode":"BICYCLING", "met_equivalent":"BICYCLING", "kgCo2PerKm": 0},
    {"value":"bikeshare", "baseMode":"BICYCLING", "met_equivalent":"BICYCLING", "kgCo2PerKm": 0},
    {"value":"scootershare", "baseMode":"E_SCOOTER", "met_equivalent":"IN_VEHICLE", "kgCo2PerKm": 0.00894},
    {"value":"drove_alone", "baseMode":"CAR", "met_equivalent":"IN_VEHICLE", "kgCo2PerKm": 0.22031},
    {"value":"shared_ride", "baseMode":"CAR", "met_equivalent":"IN_VEHICLE", "kgCo2PerKm": 0.11015},
    {"value":"e_car_drove_alone", "baseMode":"E_CAR", "met_equivalent":"IN_VEHICLE", "kgCo2PerKm": 0.08216},
    {"value":"e_car_shared_ride", "baseMode":"E_CAR", "met_equivalent":"IN_VEHICLE", "kgCo2PerKm": 0.04108},
    {"value":"moped", "baseMode":"MOPED", "met_equivalent":"IN_VEHICLE", "kgCo2PerKm": 0.05555},
    {"value":"taxi", "baseMode":"TAXI", "met_equivalent":"IN_VEHICLE", "kgCo2PerKm": 0.30741},
    {"value":"bus", "baseMode":"BUS", "met_equivalent":"IN_VEHICLE", "kgCo2PerKm": 0.20727},
    {"value":"train", "baseMode":"TRAIN", "met_equivalent":"IN_VEHICLE", "kgCo2PerKm": 0.12256},
    {"value":"free_shuttle", "baseMode":"BUS", "met_equivalent":"IN_VEHICLE", "kgCo2PerKm": 0.20727},
    {"value":"air", "baseMode":"AIR", "met_equivalent":"IN_VEHICLE", "kgCo2PerKm": 0.09975},
    {"value":"not_a_trip", "baseMode":"UNKNOWN", "met_equivalent":"UNKNOWN", "kgCo2PerKm": 0},
    {"value":"other", "baseMode":"OTHER", "met_equivalent":"UNKNOWN", "kgCo2PerKm": 0}
  ],
  "PURPOSE": [
    {"value":"home"},
    {"value":"work"},
    {"value":"at_work"},
    {"value":"school"},
    {"value":"transit_transfer"},
    {"value":"shopping"},
    {"value":"meal"},
    {"value":"pick_drop_person"},
    {"value":"pick_drop_item"},
    {"value":"personal_med"},
    {"value":"access_recreation"},
    {"value":"exercise"},
    {"value":"entertainment"},
    {"value":"religious"},
    {"value":"other"}
  ],
  "translations": {
    "en": {
      "walk": "Walk",
      "e-bike": "E-bike",
      "bike": "Regular Bike",
      "bikeshare": "Bikeshare",
      "scootershare": "Scooter share",
      "drove_alone": "Gas Car Drove Alone",
      "shared_ride": "Gas Car Shared Ride",
      "e_car_drove_alone": "E-Car Drove Alone",
      "e_car_shared_ride": "E-Car Shared Ride",
      "moped": "Moped",
      "taxi": "Taxi/Uber/Lyft",
      "bus": "Bus",
      "train": "Train",
      "free_shuttle": "Free Shuttle",
      "air": "Air",
      "not_a_trip": "Not a trip",
      "home": "Home",
      "work": "To Work",
      "at_work": "At Work",
      "school": "School",
      "transit_transfer": "Transit transfer",
      "shopping": "Shopping",
      "meal": "Meal",
      "pick_drop_person": "Pick-up/ Drop off Person",
      "pick_drop_item": "Pick-up/ Drop off Item",
      "personal_med": "Personal/ Medical",
      "access_recreation": "Access Recreation",
      "exercise": "Recreation/ Exercise",
      "entertainment": "Entertainment/ Social",
      "religious": "Religious",
      "other": "Other"
    },
    "es": {
      "walk": "Caminando",
      "e-bike": "e-bicicleta",
      "bike": "Bicicleta",
      "bikeshare": "Bicicleta compartida",
      "scootershare": "Motoneta compartida",
      "drove_alone": "Coche de Gas, Condujo solo",
      "shared_ride": "Coche de Gas, Condujo con otros",
      "e_car_drove_alone": "e-coche, Condujo solo",
      "e_car_shared_ride": "e-coche, Condujo con ontras",
      "moped": "Ciclomotor",
      "taxi": "Taxi/Uber/Lyft",
      "bus": "Autobús",
      "train": "Tren",
      "free_shuttle": "Colectivo gratuito",
      "air": "Avión",
      "not_a_trip": "No es un viaje",
      "home": "Inicio",
      "work": "Trabajo",
      "at_work": "En el trabajo",
      "school": "Escuela",
      "transit_transfer": "Transbordo",
      "shopping": "Compras",
      "meal": "Comida",
      "pick_drop_person": "Recoger/ Entregar Individuo",
      "pick_drop_item": "Recoger/ Entregar Objeto",
      "personal_med": "Personal/ Médico",
      "access_recreation": "Acceder a Recreación",
      "exercise": "Recreación/ Ejercicio",
      "entertainment": "Entretenimiento/ Social",
      "religious": "Religioso",
      "other": "Otros"
    }
  }
}
Generic_Mode_AfterMopedLabel

For generic_metrics notebook - Miles per chosen transport mode (mode_confirm:"moped").
Miles_Per_Chosen

Changed the mode_confirm back to "walk" using the following code (for comparison with the above chart).

var result_moped = db.Stage_analysis_timeseries.find({"data.user_input.mode_confirm":"moped"}, {"_id": 1})

var resultArray = result_moped.toArray();
for (var i = 0; i < resultArray.length; i++) {     var doc = resultArray[i];     var docId = doc._id;     var updateResult = db.Stage_analysis_timeseries.update(         { "_id": docId, "data.user_input.mode_confirm": "moped" },         { "$set": { "data.user_input.mode_confirm": "walk" } }     );     if (updateResult.nModified > 0) {         print(`Updated document with _id: ${docId}`);     } }

db.Stage_analysis_timeseries.find({"data.user_input.mode_confirm":"walk"}).count()
4359

> db.Stage_analysis_timeseries.find({"data.user_input.mode_confirm":"moped"}).count()
0

For generic_metrics notebook - Miles per chosen transport mode (mode_confirm:"walk")
Miles_Per_Chosen_ForWalk

These show that the "moped" label has been properly shown on the pie-chart representation.

docker-compose.dev.yml Outdated Show resolved Hide resolved
viz_scripts/bin/generate_plots.py Outdated Show resolved Hide resolved
@shankari
Copy link
Contributor

For generic_metrics notebook - Miles per chosen transport mode.

For the record, I don't see the old "walk" value for comparison in this figure.

    show dbs

    use Stage_database
    switched to db Stage_database

For the record, this is using the mongo shell. This works and I don't think you should change this now.
But you should get familiar with accessing mongo through the timseries interface to be consistent with the rest of the OpenPATH code.

@iantei
Copy link
Contributor Author

iantei commented Sep 18, 2023

For generic_metrics notebook - Miles per chosen transport mode.

For the record, I don't see the old "walk" value for comparison in this figure.

I have updated the above comment with a figure for the "walk" value for comparative reference to "moped" value for "Miles per chosen transport mode".

…ted changes for DYNAMIC_CONFIG back to STUDY_CONFIG from generate_plots.py
docker-compose.dev.yml Outdated Show resolved Hide resolved
Copy link
Contributor

@shankari shankari left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Almost there - please clean up the whitespace and extraneous changes as below.

Also, I don't understand the reason for the difference between these:

Before any changes (1) walk -> moped, default mapping (2) walk -> moped, custom mapping (3)
image image image

I can understand the subtle difference between (1) and (2), since the "moped" and the original "other" get merged. But why are the percentages for (3) so different from (1) and (2)?

viz_scripts/bin/generate_plots.py Outdated Show resolved Hide resolved
viz_scripts/plots.py Outdated Show resolved Hide resolved
viz_scripts/plots.py Outdated Show resolved Hide resolved
viz_scripts/plots.py Outdated Show resolved Hide resolved
viz_scripts/scaffolding.py Show resolved Hide resolved
…g_labels() function. Refactored compute_CO2_impact_dynamic() and compute_CO2_footprint_dynamic() to compute the CO2 emission in compute_CO2_footprint_dynamic() function and differentiate between Mode_confirm and Replaced_mode CO2 emission in compute_CO2_impact_dynamic().
…l_lb_CO2 emission in Dynamic Label case, and total_kg_CO2_emission in default config case.
…avoid code duplication for dynamic labels and default mapping cases.
…d prefix to the new functions to reflect its functionality.
@iantei
Copy link
Contributor Author

iantei commented Nov 7, 2023

Executed the following testing for regression testing with default mapping
STUDY_CONFIG=stage-program

Default Mapping: Automated notebook execution for `energy_calculations` and `generic_timeseries` notebook.

Energy_calculations.ipynb

(emission) root@651667db60c6:/usr/src/app/saved-notebooks# PYTHONPATH=.. python bin/generate_plots.py energy_calculations.ipynb default
/usr/src/app/saved-notebooks/bin/generate_plots.py:30: SyntaxWarning: "is not" with a literal. Did you mean "!="?
  if r.status_code is not 200:
About to download config from https://raw.githubusercontent.com/e-mission/nrel-openpath-deploy-configs/main/configs/stage-program.nrel-op.json
Successfully downloaded config with version 1 for Staging environment for testing programs only and data collection URL https://openpath-stage.nrel.gov/api/
label_options is unavailable for the dynamic_config in stage-program
Running at 2023-11-07T19:54:02.870477+00:00 with args Namespace(plot_notebook='energy_calculations.ipynb', program='default', date=None) for range (<Arrow [2020-09-01T00:00:00+00:00]>, <Arrow [2023-11-01T00:00:00+00:00]>)
Running at 2023-11-07T19:54:02.973217+00:00 with params [Parameter('year', int), Parameter('month', int), Parameter('program', str, value='default'), Parameter('study_type', str, value='program'), Parameter('mode_of_interest', str, value='e-bike'), Parameter('include_test_users', bool, value=True), Parameter('is_debug_mode', bool, value=False), Parameter('dynamic_labels', dict, value={})]

Generic_timeseries.ipynb:

(emission) root@651667db60c6:/usr/src/app/saved-notebooks# PYTHONPATH=.. python bin/generate_plots.py generic_timeseries.ipynb default
/usr/src/app/saved-notebooks/bin/generate_plots.py:30: SyntaxWarning: "is not" with a literal. Did you mean "!="?
  if r.status_code is not 200:
About to download config from https://raw.githubusercontent.com/e-mission/nrel-openpath-deploy-configs/main/configs/stage-program.nrel-op.json
Successfully downloaded config with version 1 for Staging environment for testing programs only and data collection URL https://openpath-stage.nrel.gov/api/
label_options is unavailable for the dynamic_config in stage-program
Running at 2023-11-07T20:35:56.112032+00:00 with args Namespace(plot_notebook='generic_timeseries.ipynb', program='default', date=None) for range (<Arrow [2020-09-01T00:00:00+00:00]>, <Arrow [2023-11-01T00:00:00+00:00]>)
Running at 2023-11-07T20:35:56.152280+00:00 with params [Parameter('year', int), Parameter('month', int), Parameter('program', str, value='default'), Parameter('study_type', str, value='program'), Parameter('include_test_users', bool, value=True), Parameter('dynamic_labels', dict, value={})]
Running at 2023-11-07T20:36:15.883235+00:00 with params [Parameter('year', int, value=2020), Parameter('month', int, value=9), Parameter('program', str, value='default'), Parameter('study_type', str, value='program'), Parameter('include_test_users', bool, value=True), Parameter('dynamic_labels', dict, value={})]

Results:

Timeseries of emissions All Data Timeseries of emissions per mile All Data
image image

Dynamic Labels: Automated notebook execution for `energy_calculations` and `generic_timeseries` notebook

Execution:

Executed the following testing for regression testing with default mapping
STUDY_CONFIG=dev-emulator-program
Loaded the web page using `http://localhost:3274/?study_config=dev-emulator-program`

Results:

Timeseries of emissions Timeseries of emissions per
image image

@iantei
Copy link
Contributor Author

iantei commented Nov 7, 2023

The math still seems to be wrong.

I have given justification for the math I am using for the computation of CO2 emission - which is lb_CO2/ trip or kg_CO2/trip. Please let me know if that adds up rightfully.

And you have too much copy-pasted code.

Yes, I did realize that. I have re-factored both generic_timeseries and energy_calculations notebooks to avoid code duplication while keeping the functionality intact.

@shankari
Copy link
Contributor

@iantei you need to fix your markdown. I have edited #91 (comment) and #91 (comment), for example, to be correct.

viz_scripts/generic_timeseries.ipynb Outdated Show resolved Hide resolved
" \n",
" # Add 7-day rolling avg smoothing to better see trends\n",
" mode_counts['trip_count_smooth'] = mode_counts.groupby(['user_id','Mode_confirm'])['trip_count'].apply(lambda x: x.rolling(7,1).mean())\n",
" mode_distance[f'{distance_unit}smooth'] = mode_distance.groupby(['user_id','Mode_confirm'])[f'{distance_unit}'].apply(lambda x: x.rolling(7,1).mean())\n",
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I don't think this is actually correct. How did you test this? I don't see how it can work without the underscore.

Suggested change
" mode_distance[f'{distance_unit}smooth'] = mode_distance.groupby(['user_id','Mode_confirm'])[f'{distance_unit}'].apply(lambda x: x.rolling(7,1).mean())\n",
" mode_distance[f'{distance_unit}_smooth'] = mode_distance.groupby(['user_id','Mode_confirm'])[f'{distance_unit}'].apply(lambda x: x.rolling(7,1).mean())\n",

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I have tested the generic_timeseries notebook with both default mapping and dynamic config through running Jupyter notebook automatically as mentioned above.
Interestingly, we don't use mode_distance after we have called mode_distance[f'{distance_unit}_smooth']. This is an existing code changes. I am not sure if this was written to handle future things, but at the moment, it does not execute anything over mode_distance.
Still, updated the above code to use mode_distance[f'{distance_unit}_smooth'].

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We do use it to compute "Emissions per mile per day"

try:
    # Emissions per mile per day across all users (travel efficiency)
    # Note that the energy plot will be identical to this one since scale factor is divided out
    emissions['CO2_per_mile'] = emissions['Mode_confirm_lb_CO2'] / emissions['distance_miles_smooth']
    emissions['CO2_per_mile'] = emissions['CO2_per_mile'].fillna(0)
    plot_data = emissions.groupby(['date_time'])['CO2_per_mile'].agg(['mean']).reset_index()

Is this not in the new codebase?

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Ah I see, that uses emissions['distance_miles_smooth'] and not mode_distance[f'{distance_unit}_smooth']

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It is used in mode_specific_timeseries, though

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

And I don't see any related changes in mode_specific_timeseries. Ah but that is because mode_specific_timeseries does not appear to have any code related to emission calculations.
@iantei I would like to see such investigation and clarification/documentation in the future.

viz_scripts/generic_timeseries.ipynb Outdated Show resolved Hide resolved
Copy link
Contributor

@shankari shankari left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The only correctness issue here is the one with the underscore. But since you have to fix that anyway, you might as well fix the formatting/cleanliness issues as well.

viz_scripts/scaffolding.py Outdated Show resolved Hide resolved
viz_scripts/scaffolding.py Outdated Show resolved Hide resolved
Comment on lines 429 to 434
def print_CO2_emission_calculations(data_eb, ebco2_lb, ebco2_kg, dynamic_labels):

filtered_taxi_data = data_eb[data_eb['Replaced_mode'] == "Taxi/Uber/Lyft"]
filtered_bus_data = data_eb[data_eb['Replaced_mode'] == "Bus"]
filtered_freeshuttle_data = data_eb[data_eb['Replaced_mode'] == "Free Shuttle"]
filtered_walk_data = data_eb[data_eb['Replaced_mode'] == "Walk"]
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Please add some notes here about the particular dynamic config that this works with. Because we have at least two production (non-example) dynamic configs now (Laos and CA e-bike) and the second is even in English. So people might be confused while debugging and not know why this doesn't work.

…CO2_footprint_default() and compute_CO2_footprint_dynamic().
@iantei
Copy link
Contributor Author

iantei commented Nov 13, 2023

Accounting to above changes the generic_timeseries notebook has been tested for regression testing in both default mapping and dynamic config cases.

Notebook: `Generic Timeseries` with STUDY_CONFIG=`dev-emulator-program` | Dynamic Config

Steps executed:

(emission) root@dac7d8c525e4:/usr/src/app/saved-notebooks# PYTHONPATH=.. python bin/update_mappings.py mapping_dictionaries.ipynb
(emission) root@dac7d8c525e4:/usr/src/app/saved-notebooks# PYTHONPATH=.. python bin/generate_plots.py generic_timeseries.ipynb default
/usr/src/app/saved-notebooks/bin/generate_plots.py:30: SyntaxWarning: "is not" with a literal. Did you mean "!="?
  if r.status_code is not 200:
About to download config from https://raw.githubusercontent.com/e-mission/nrel-openpath-deploy-configs/main/configs/dev-emulator-program.nrel-op.json
Successfully downloaded config with version 1 for Development environment (program) and data collection URL default
Dynamic labels download was successful for nrel-openpath-deploy-configs: dev-emulator-program
Running at 2023-11-13T21:21:36.391448+00:00 with args Namespace(plot_notebook='generic_timeseries.ipynb', program='default', date=None) for range (<Arrow [2020-09-01T00:00:00+00:00]>, <Arrow [2023-11-01T00:00:00+00:00]>)
Running at 2023-11-13T21:21:36.428724+00:00 with params [Parameter('year', int), Parameter('month', int), Parameter('program', str, value='default'), Parameter('study_type', str, value='program'), Parameter('include_test_users', bool, value=False), Parameter('dynamic_labels', dict, value={'MODE': [{'value': 'walk', 'baseMode': 'WALKING', 'met_equivalent': 'WALKING', 'kgCo2PerKm': 0}, {'value': 'e-bike', 'baseMode': 'E_BIKE', 'met': {'ALL': {'range': [0, -1], 'mets': 4.9}}, 'kgCo2PerKm': 0.00728}, {'value': 'bike', 'baseMode': 'BICYCLING', 'met_equivalent': 'BICYCLING', 'kgCo2PerKm': 0}, {'value': 'bikeshare', 'baseMode': 'BICYCLING', 'met_equivalent': 'BICYCLING', 'kgCo2PerKm': 0}, {'value': 'scootershare', 'baseMode': 'E_SCOOTER', 'met_equivalent': 'IN_VEHICLE', 'kgCo2PerKm': 0.00894}, {'value': 'drove_alone', 'baseMode': 'CAR', 'met_equivalent': 'IN_VEHICLE', 'kgCo2PerKm': 0.22031}, {'value': 'shared_ride', 'baseMode': 'CAR', 'met_equivalent': 'IN_VEHICLE', 'kgCo2PerKm': 0.11015}, {'value': 'e_car_drove_alone', 'baseMode': 'E_CAR', 'met_equivalent': 'IN_VEHICLE', 'kgCo2PerKm': 0.08216}, {'value': 'e_car_shared_ride', 'baseMode': 'E_CAR', 'met_equivalent': 'IN_VEHICLE', 'kgCo2PerKm': 0.04108}, {'value': 'moped', 'baseMode': 'MOPED', 'met_equivalent': 'IN_VEHICLE', 'kgCo2PerKm': 0.05555}, {'value': 'taxi', 'baseMode': 'TAXI', 'met_equivalent': 'IN_VEHICLE', 'kgCo2PerKm': 0.30741}, {'value': 'bus', 'baseMode': 'BUS', 'met_equivalent': 'IN_VEHICLE', 'kgCo2PerKm': 0.20727}, {'value': 'train', 'baseMode': 'TRAIN', 'met_equivalent': 'IN_VEHICLE', 'kgCo2PerKm': 0.12256}, {'value': 'free_shuttle', 'baseMode': 'BUS', 'met_equivalent': 'IN_VEHICLE', 'kgCo2PerKm': 0.20727}, {'value': 'air', 'baseMode': 'AIR', 'met_equivalent': 'IN_VEHICLE', 'kgCo2PerKm': 0.09975}, {'value': 'not_a_trip', 'baseMode': 'UNKNOWN', 'met_equivalent': 'UNKNOWN', 'kgCo2PerKm': 0}, {'value': 'other', 'baseMode': 'OTHER', 'met_equivalent': 'UNKNOWN', 'kgCo2PerKm': 0}], 'PURPOSE': [{'value': 'home'}, {'value': 'work'}, {'value': 'at_work'}, {'value': 'school'}, {'value': 'transit_transfer'}, {'value': 'shopping'}, {'value': 'meal'}, {'value': 'pick_drop_person'}, {'value': 'pick_drop_item'}, {'value': 'personal_med'}, {'value': 'access_recreation'}, {'value': 'exercise'}, {'value': 'entertainment'}, {'value': 'religious'}, {'value': 'other'}], 'REPLACED_MODE': [{'value': 'no_travel'}, {'value': 'walk'}, {'value': 'bike'}, {'value': 'bikeshare'}, {'value': 'scootershare'}, {'value': 'drove_alone'}, {'value': 'shared_ride'}, {'value': 'e_car_drove_alone'}, {'value': 'e_car_shared_ride'}, {'value': 'taxi'}, {'value': 'bus'}, {'value': 'train'}, {'value': 'free_shuttle'}, {'value': 'other'}], 'translations': {'en': {'walk': 'Walk', 'e-bike': 'E-bike', 'bike': 'Regular Bike', 'bikeshare': 'Bikeshare', 'scootershare': 'Scooter share', 'drove_alone': 'Gas Car Drove Alone', 'shared_ride': 'Gas Car Shared Ride', 'e_car_drove_alone': 'E-Car Drove Alone', 'e_car_shared_ride': 'E-Car Shared Ride', 'moped': 'Moped', 'taxi': 'Taxi/Uber/Lyft', 'bus': 'Bus', 'train': 'Train', 'free_shuttle': 'Free Shuttle', 'air': 'Air', 'not_a_trip': 'Not a trip', 'no_travel': 'No travel', 'home': 'Home', 'work': 'To Work', 'at_work': 'At Work', 'school': 'School', 'transit_transfer': 'Transit transfer', 'shopping': 'Shopping', 'meal': 'Meal', 'pick_drop_person': 'Pick-up/ Drop off Person', 'pick_drop_item': 'Pick-up/ Drop off Item', 'personal_med': 'Personal/ Medical', 'access_recreation': 'Access Recreation', 'exercise': 'Recreation/ Exercise', 'entertainment': 'Entertainment/ Social', 'religious': 'Religious', 'other': 'Other'}, 'es': {'walk': 'Caminando', 'e-bike': 'e-bicicleta', 'bike': 'Bicicleta', 'bikeshare': 'Bicicleta compartida', 'scootershare': 'Motoneta compartida', 'drove_alone': 'Coche de Gas, Condujo solo', 'shared_ride': 'Coche de Gas, Condujo con otros', 'e_car_drove_alone': 'e-coche, Condujo solo', 'e_car_shared_ride': 'e-coche, Condujo con ontras', 'moped': 'Ciclomotor', 'taxi': 'Taxi/Uber/Lyft', 'bus': 'Autobús', 'train': 'Tren', 'free_shuttle': 'Colectivo gratuito', 'air': 'Avión', 'not_a_trip': 'No es un viaje', 'no_travel': 'No viajar', 'home': 'Inicio', 'work': 'Trabajo', 'at_work': 'En el trabajo', 'school': 'Escuela', 'transit_transfer': 'Transbordo', 'shopping': 'Compras', 'meal': 'Comida', 'pick_drop_person': 'Recoger/ Entregar Individuo', 'pick_drop_item': 'Recoger/ Entregar Objeto', 'personal_med': 'Personal/ Médico', 'access_recreation': 'Acceder a Recreación', 'exercise': 'Recreación/ Ejercicio', 'entertainment': 'Entretenimiento/ Social', 'religious': 'Religioso', 'other': 'Otros'}}})]
Running at 2023-11-13T21:22:00.189112+00:00 with params [Parameter('year', int, value=2020), Parameter('month', int, value=9), Parameter('program', str, value='default'), Parameter('study_type', str, value='program'), Parameter('include_test_users', bool, value=False), Parameter('dynamic_labels', dict, value={'MODE': [{'value': 'walk', 'baseMode': 'WALKING', 'met_equivalent': 'WALKING', 'kgCo2PerKm': 0}, {'value': 'e-bike', 'baseMode': 'E_BIKE', 'met': {'ALL': {'range': [0, -1], 'mets': 4.9}}, 'kgCo2PerKm': 0.00728}, {'value': 'bike', 'baseMode': 'BICYCLING', 'met_equivalent': 'BICYCLING', 'kgCo2PerKm': 0}, {'value': 'bikeshare', 'baseMode': 'BICYCLING', 'met_equivalent': 'BICYCLING', 'kgCo2PerKm': 0}, {'value': 'scootershare', 'baseMode': 'E_SCOOTER', 'met_equivalent': 'IN_VEHICLE', 'kgCo2PerKm': 0.00894}, {'value': 'drove_alone', 'baseMode': 'CAR', 'met_equivalent': 'IN_VEHICLE', 'kgCo2PerKm': 0.22031}, {'value': 'shared_ride', 'baseMode': 'CAR', 'met_equivalent': 'IN_VEHICLE', 'kgCo2PerKm': 0.11015}, {'value': 'e_car_drove_alone', 'baseMode': 'E_CAR', 'met_equivalent': 'IN_VEHICLE', 'kgCo2PerKm': 0.08216}, {'value': 'e_car_shared_ride', 'baseMode': 'E_CAR', 'met_equivalent': 'IN_VEHICLE', 'kgCo2PerKm': 0.04108}, {'value': 'moped', 'baseMode': 'MOPED', 'met_equivalent': 'IN_VEHICLE', 'kgCo2PerKm': 0.05555}, {'value': 'taxi', 'baseMode': 'TAXI', 'met_equivalent': 'IN_VEHICLE', 'kgCo2PerKm': 0.30741}, {'value': 'bus', 'baseMode': 'BUS', 'met_equivalent': 'IN_VEHICLE', 'kgCo2PerKm': 0.20727}, {'value': 'train', 'baseMode': 'TRAIN', 'met_equivalent': 'IN_VEHICLE', 'kgCo2PerKm': 0.12256}, {'value': 'free_shuttle', 'baseMode': 'BUS', 'met_equivalent': 'IN_VEHICLE', 'kgCo2PerKm': 0.20727}, {'value': 'air', 'baseMode': 'AIR', 'met_equivalent': 'IN_VEHICLE', 'kgCo2PerKm': 0.09975}, {'value': 'not_a_trip', 'baseMode': 'UNKNOWN', 'met_equivalent': 'UNKNOWN', 'kgCo2PerKm': 0}, {'value': 'other', 'baseMode': 'OTHER', 'met_equivalent': 'UNKNOWN', 'kgCo2PerKm': 0}], 'PURPOSE': [{'value': 'home'}, {'value': 'work'}, {'value': 'at_work'}, {'value': 'school'}, {'value': 'transit_transfer'}, {'value': 'shopping'}, {'value': 'meal'}, {'value': 'pick_drop_person'}, {'value': 'pick_drop_item'}, {'value': 'personal_med'}, {'value': 'access_recreation'}, {'value': 'exercise'}, {'value': 'entertainment'}, {'value': 'religious'}, {'value': 'other'}], 'REPLACED_MODE': [{'value': 'no_travel'}, {'value': 'walk'}, {'value': 'bike'}, {'value': 'bikeshare'}, {'value': 'scootershare'}, {'value': 'drove_alone'}, {'value': 'shared_ride'}, {'value': 'e_car_drove_alone'}, {'value': 'e_car_shared_ride'}, {'value': 'taxi'}, {'value': 'bus'}, {'value': 'train'}, {'value': 'free_shuttle'}, {'value': 'other'}], 'translations': {'en': {'walk': 'Walk', 'e-bike': 'E-bike', 'bike': 'Regular Bike', 'bikeshare': 'Bikeshare', 'scootershare': 'Scooter share', 'drove_alone': 'Gas Car Drove Alone', 'shared_ride': 'Gas Car Shared Ride', 'e_car_drove_alone': 'E-Car Drove Alone', 'e_car_shared_ride': 'E-Car Shared Ride', 'moped': 'Moped', 'taxi': 'Taxi/Uber/Lyft', 'bus': 'Bus', 'train': 'Train', 'free_shuttle': 'Free Shuttle', 'air': 'Air', 'not_a_trip': 'Not a trip', 'no_travel': 'No travel', 'home': 'Home', 'work': 'To Work', 'at_work': 'At Work', 'school': 'School', 'transit_transfer': 'Transit transfer', 'shopping': 'Shopping', 'meal': 'Meal', 'pick_drop_person': 'Pick-up/ Drop off Person', 'pick_drop_item': 'Pick-up/ Drop off Item', 'personal_med': 'Personal/ Medical', 'access_recreation': 'Access Recreation', 'exercise': 'Recreation/ Exercise', 'entertainment': 'Entertainment/ Social', 'religious': 'Religious', 'other': 'Other'}, 'es': {'walk': 'Caminando', 'e-bike': 'e-bicicleta', 'bike': 'Bicicleta', 'bikeshare': 'Bicicleta compartida', 'scootershare': 'Motoneta compartida', 'drove_alone': 'Coche de Gas, Condujo solo', 'shared_ride': 'Coche de Gas, Condujo con otros', 'e_car_drove_alone': 'e-coche, Condujo solo', 'e_car_shared_ride': 'e-coche, Condujo con ontras', 'moped': 'Ciclomotor', 'taxi': 'Taxi/Uber/Lyft', 'bus': 'Autobús', 'train': 'Tren', 'free_shuttle': 'Colectivo gratuito', 'air': 'Avión', 'not_a_trip': 'No es un viaje', 'no_travel': 'No viajar', 'home': 'Inicio', 'work': 'Trabajo', 'at_work': 'En el trabajo', 'school': 'Escuela', 'transit_transfer': 'Transbordo', 'shopping': 'Compras', 'meal': 'Comida', 'pick_drop_person': 'Recoger/ Entregar Individuo', 'pick_drop_item': 'Recoger/ Entregar Objeto', 'personal_med': 'Personal/ Médico', 'access_recreation': 'Acceder a Recreación', 'exercise': 'Recreación/ Ejercicio', 'entertainment': 'Entretenimiento/ Social', 'religious': 'Religioso', 'other': 'Otros'}}})]
Running at 2023-11-13T21:22:05.560116+00:00 with params [Parameter('year', int, value=2020), Parameter('month', int, value=10), Parameter('program', str, value='default'), Parameter('study_type', str, value='program'), Parameter('include_test_users', bool, value=False), Parameter('dynamic_labels', dict, value={'MODE': [{'value': 'walk', 'baseMode': 'WALKING', 'met_equivalent': 'WALKING', 'kgCo2PerKm': 0}, {'value': 'e-bike', 'baseMode': 'E_BIKE', 'met': {'ALL': {'range': [0, -1], 'mets': 4.9}}, 'kgCo2PerKm': 0.00728}, {'value': 'bike', 'baseMode': 'BICYCLING', 'met_equivalent': 'BICYCLING', 'kgCo2PerKm': 0}, {'value': 'bikeshare', 'baseMode': 'BICYCLING', 'met_equivalent': 'BICYCLING', 'kgCo2PerKm': 0}, {'value': 'scootershare', 'baseMode': 'E_SCOOTER', 'met_equivalent': 'IN_VEHICLE', 'kgCo2PerKm': 0.00894}, {'value': 'drove_alone', 'baseMode': 'CAR', 'met_equivalent': 'IN_VEHICLE', 'kgCo2PerKm': 0.22031}, {'value': 'shared_ride', 'baseMode': 'CAR', 'met_equivalent': 'IN_VEHICLE', 'kgCo2PerKm': 0.11015}, {'value': 'e_car_drove_alone', 'baseMode': 'E_CAR', 'met_equivalent': 'IN_VEHICLE', 'kgCo2PerKm': 0.08216}, {'value': 'e_car_shared_ride', 'baseMode': 'E_CAR', 'met_equivalent': 'IN_VEHICLE', 'kgCo2PerKm': 0.04108}, {'value': 'moped', 'baseMode': 'MOPED', 'met_equivalent': 'IN_VEHICLE', 'kgCo2PerKm': 0.05555}, {'value': 'taxi', 'baseMode': 'TAXI', 'met_equivalent': 'IN_VEHICLE', 'kgCo2PerKm': 0.30741}, {'value': 'bus', 'baseMode': 'BUS', 'met_equivalent': 'IN_VEHICLE', 'kgCo2PerKm': 0.20727}, {'value': 'train', 'baseMode': 'TRAIN', 'met_equivalent': 'IN_VEHICLE', 'kgCo2PerKm': 0.12256}, {'value': 'free_shuttle', 'baseMode': 'BUS', 'met_equivalent': 'IN_VEHICLE', 'kgCo2PerKm': 0.20727}, {'value': 'air', 'baseMode': 'AIR', 'met_equivalent': 'IN_VEHICLE', 'kgCo2PerKm': 0.09975}, {'value': 'not_a_trip', 'baseMode': 'UNKNOWN', 'met_equivalent': 'UNKNOWN', 'kgCo2PerKm': 0}, {'value': 'other', 'baseMode': 'OTHER', 'met_equivalent': 'UNKNOWN', 'kgCo2PerKm': 0}], 'PURPOSE': [{'value': 'home'}, {'value': 'work'}, {'value': 'at_work'}, {'value': 'school'}, {'value': 'transit_transfer'}, {'value': 'shopping'}, {'value': 'meal'}, {'value': 'pick_drop_person'}, {'value': 'pick_drop_item'}, {'value': 'personal_med'}, {'value': 'access_recreation'}, {'value': 'exercise'}, {'value': 'entertainment'}, {'value': 'religious'}, {'value': 'other'}], 'REPLACED_MODE': [{'value': 'no_travel'}, {'value': 'walk'}, {'value': 'bike'}, {'value': 'bikeshare'}, {'value': 'scootershare'}, {'value': 'drove_alone'}, {'value': 'shared_ride'}, {'value': 'e_car_drove_alone'}, {'value': 'e_car_shared_ride'}, {'value': 'taxi'}, {'value': 'bus'}, {'value': 'train'}, {'value': 'free_shuttle'}, {'value': 'other'}], 'translations': {'en': {'walk': 'Walk', 'e-bike': 'E-bike', 'bike': 'Regular Bike', 'bikeshare': 'Bikeshare', 'scootershare': 'Scooter share', 'drove_alone': 'Gas Car Drove Alone', 'shared_ride': 'Gas Car Shared Ride', 'e_car_drove_alone': 'E-Car Drove Alone', 'e_car_shared_ride': 'E-Car Shared Ride', 'moped': 'Moped', 'taxi': 'Taxi/Uber/Lyft', 'bus': 'Bus', 'train': 'Train', 'free_shuttle': 'Free Shuttle', 'air': 'Air', 'not_a_trip': 'Not a trip', 'no_travel': 'No travel', 'home': 'Home', 'work': 'To Work', 'at_work': 'At Work', 'school': 'School', 'transit_transfer': 'Transit transfer', 'shopping': 'Shopping', 'meal': 'Meal', 'pick_drop_person': 'Pick-up/ Drop off Person', 'pick_drop_item': 'Pick-up/ Drop off Item', 'personal_med': 'Personal/ Medical', 'access_recreation': 'Access Recreation', 'exercise': 'Recreation/ Exercise', 'entertainment': 'Entertainment/ Social', 'religious': 'Religious', 'other': 'Other'}, 'es': {'walk': 'Caminando', 'e-bike': 'e-bicicleta', 'bike': 'Bicicleta', 'bikeshare': 'Bicicleta compartida', 'scootershare': 'Motoneta compartida', 'drove_alone': 'Coche de Gas, Condujo solo', 'shared_ride': 'Coche de Gas, Condujo con otros', 'e_car_drove_alone': 'e-coche, Condujo solo', 'e_car_shared_ride': 'e-coche, Condujo con ontras', 'moped': 'Ciclomotor', 'taxi': 'Taxi/Uber/Lyft', 'bus': 'Autobús', 'train': 'Tren', 'free_shuttle': 'Colectivo gratuito', 'air': 'Avión', 'not_a_trip': 'No es un viaje', 'no_travel': 'No viajar', 'home': 'Inicio', 'work': 'Trabajo', 'at_work': 'En el trabajo', 'school': 'Escuela', 'transit_transfer': 'Transbordo', 'shopping': 'Compras', 'meal': 'Comida', 'pick_drop_person': 'Recoger/ Entregar Individuo', 'pick_drop_item': 'Recoger/ Entregar Objeto', 'personal_med': 'Personal/ Médico', 'access_recreation': 'Acceder a Recreación', 'exercise': 'Recreación/ Ejercicio', 'entertainment': 'Entretenimiento/ Social', 'religious': 'Religioso', 'other': 'Otros'}}})]

Notebook: `Generic Timeseries` with STUDY_CONFIG=`stage-program` | Default Mapping

Execution steps:

(emission) root@52bab48d1dd6:/usr/src/app/saved-notebooks# PYTHONPATH=.. python bin/generate_plots.py generic_timeseries.ipynb default
/usr/src/app/saved-notebooks/bin/generate_plots.py:30: SyntaxWarning: "is not" with a literal. Did you mean "!="?
  if r.status_code is not 200:
About to download config from https://raw.githubusercontent.com/e-mission/nrel-openpath-deploy-configs/main/configs/stage-program.nrel-op.json
Successfully downloaded config with version 1 for Staging environment for testing programs only and data collection URL https://openpath-stage.nrel.gov/api/
label_options is unavailable for the dynamic_config in stage-program
Running at 2023-11-13T21:47:40.574469+00:00 with args Namespace(plot_notebook='generic_timeseries.ipynb', program='default', date=None) for range (<Arrow [2020-09-01T00:00:00+00:00]>, <Arrow [2023-11-01T00:00:00+00:00]>)
Running at 2023-11-13T21:47:40.607876+00:00 with params [Parameter('year', int), Parameter('month', int), Parameter('program', str, value='default'), Parameter('study_type', str, value='program'), Parameter('include_test_users', bool, value=True), Parameter('dynamic_labels', dict, value={})]
Running at 2023-11-13T21:48:05.833069+00:00 with params [Parameter('year', int, value=2020), Parameter('month', int, value=9), Parameter('program', str, value='default'), Parameter('study_type', str, value='program'), Parameter('include_test_users', bool, value=True), Parameter('dynamic_labels', dict, value={})]
Running at 2023-11-13T21:48:11.220470+00:00 with params [Parameter('year', int, value=2020), Parameter('month', int, value=10), Parameter('program', str, value='default'), Parameter('study_type', str, value='program'), Parameter('include_test_users', bool, value=True), Parameter('dynamic_labels', dict, value={})]
Running at 2023-11-13T21:48:16.596462+00:00 with params [Parameter('year', int, value=2020), Parameter('month', int, value=11), Parameter('program', str, value='default'), Parameter('study_type', str, value='program'), Parameter('include_test_users', bool, value=True), Parameter('dynamic_labels', dict, value={})]
Running at 2023-11-13T21:48:21.795305+00:00 with params [Parameter('year', int, value=2020), Parameter('month', int, value=12), Parameter('program', str, value='default'), Parameter('study_type', str, value='program'), Parameter('include_test_users', bool, value=True), Parameter('dynamic_labels', dict, value={})]
Running at 2023-11-13T21:48:28.244178+00:00 with params [Parameter('year', int, value=2021), Parameter('month', int, value=1), Parameter('program', str, value='default'), Parameter('study_type', str, value='program'), Parameter('include_test_users', bool, value=True), Parameter('dynamic_labels', dict, value={})]
Running at 2023-11-13T21:48:34.418982+00:00 with params [Parameter('year', int, value=2021), Parameter('month', int, value=2), Parameter('program', str, value='default'), Parameter('study_type', str, value='program'), Parameter('include_test_users', bool, value=True), Parameter('dynamic_labels', dict, value={})]

Results comparison between Default Mapping and Dynamic Config in Generic Timeseries:

Timeseries of Emissions

Default Mapping Dynamic Config
Screenshot 2023-11-20 at 9 45 01 PM Screenshot 2023-11-20 at 10 03 58 PM

Timeseries of Emissions per mile/kilometer

Default Mapping Dynamic Config
Screenshot 2023-11-20 at 9 46 04 PM Screenshot 2023-11-20 at 10 04 04 PM

Timeseries of active users

Default Mapping Dynamic Config
Screenshot 2023-11-20 at 9 46 30 PM Screenshot 2023-11-20 at 10 04 12 PM

Timeseries of all mode shares

Default Mapping Dynamic Config
Screenshot 2023-11-20 at 9 46 48 PM Screenshot 2023-11-20 at 10 04 18 PM

@shankari
Copy link
Contributor

@iantei thank you for putting the test results into the PR as "Details".
Can you please put them side by side in a table (you can still put them in an expandable section)?
It is easiest to compare values when they are side by side so that our brains pick up on visual cues properly.
Please also include testing for the generic metrics in this final round to ensure that there are no regressions.

@shankari
Copy link
Contributor

@iantei looking through my prior comments to make sure that they are all resolved, I ran into this comment.

Similarly, notebook execution for sensed notebook failed since we have some NaN values in the dataset used - as justified above.

As justified where? The sensed notebooks should not be config specific since we don't use user labels in them.

@shankari
Copy link
Contributor

I am provisionally marking this as approved. Congratulations, it only took 2 months 😄

Before I merge, I want to see:

  • I still want to see all the outputs (e.g. generic metrics) one last time to make sure that there are no regressions
  • I want to see an explanation of the sensed notebooks in the Lao case

@iantei
Copy link
Contributor Author

iantei commented Nov 20, 2023

Comparison of generic metrics notebook outputs:

Executed the notebook "Generic Metrics" for both the STUDY_CONFIG for comparison between default mapping and dynamic config.

Steps executed:

A. For Default Mapping (STUDY_CONFIG=stage-program)

(emission) root@d7768840b5c9:/usr/src/app/saved-notebooks# PYTHONPATH=.. python bin/generate_plots.py generic_metrics.ipynb default
/usr/src/app/saved-notebooks/bin/generate_plots.py:30: SyntaxWarning: "is not" with a literal. Did you mean "!="?
  if r.status_code is not 200:
About to download config from https://raw.githubusercontent.com/e-mission/nrel-openpath-deploy-configs/main/configs/stage-program.nrel-op.json
Successfully downloaded config with version 1 for Staging environment for testing programs only and data collection URL https://openpath-stage.nrel.gov/api/
label_options is unavailable for the dynamic_config in stage-program
Running at 2023-11-20T19:11:35.554525+00:00 with args Namespace(plot_notebook='generic_metrics.ipynb', program='default', date=None) for range (<Arrow [2020-09-01T00:00:00+00:00]>, <Arrow [2023-11-01T00:00:00+00:00]>)
Running at 2023-11-20T19:11:35.592872+00:00 with params [Parameter('year', int), Parameter('month', int), Parameter('program', str, value='default'), Parameter('study_type', str, value='program'), Parameter('include_test_users', bool, value=True), Parameter('dynamic_labels', dict, value={})]
Running at 2023-11-20T19:11:51.849538+00:00 with params [Parameter('year', int, value=2020), Parameter('month', int, value=9), Parameter('program', str, value='default'), Parameter('study_type', str, value='program'), Parameter('include_test_users', bool, value=True), Parameter('dynamic_labels', dict, value={})]
Running at 2023-11-20T19:11:58.706112+00:00 with params [Parameter('year', int, value=2020), Parameter('month', int, value=10), Parameter('program', str, value='default'), Parameter('study_type', str, value='program'), Parameter('include_test_users', bool, value=True), Parameter('dynamic_labels', dict, value={})]
Running at 2023-11-20T19:12:05.421774+00:00 with params [Parameter('year', int, value=2020), Parameter('month', int, value=11), Parameter('program', str, value='default'), Parameter('study_type', str, value='program'), Parameter('include_test_users', bool, value=True), Parameter('dynamic_labels', dict, value={})]
Running at 2023-11-20T19:12:12.831446+00:00 with params [Parameter('year', int, value=2020), Parameter('month', int, value=12), Parameter('program', str, value='default'), Parameter('study_type', str, value='program'), Parameter('include_test_users', bool, value=True), Parameter('dynamic_labels', dict, value={})]

B. For Dynamic Config (STUDY_CONFIG=dev-emulator-program):

(emission) root@4f0d3eb39c2e:/usr/src/app/saved-notebooks# ls
Dockerfile                          auxiliary_files  docker                     generic_metrics_sensed.ipynb  mode_specific_metrics.ipynb     run_from_host   variation_across_individuals.ipynb
Program_trip_characteristics.ipynb  bin              energy_calculations.ipynb  generic_timeseries.ipynb      mode_specific_timeseries.ipynb  scaffolding.py
__pycache__                         conf             generic_metrics.ipynb      mapping_dictionaries.ipynb    plots.py                        tests
(emission) root@4f0d3eb39c2e:/usr/src/app/saved-notebooks# PYTHONPATH=.. python bin/update_mappings.py mapping_dictionaries.ipynb
(emission) root@4f0d3eb39c2e:/usr/src/app/saved-notebooks# PYTHONPATH=.. python bin/generate_plots.py generic_metrics.ipynb default
/usr/src/app/saved-notebooks/bin/generate_plots.py:30: SyntaxWarning: "is not" with a literal. Did you mean "!="?
  if r.status_code is not 200:
About to download config from https://raw.githubusercontent.com/e-mission/nrel-openpath-deploy-configs/main/configs/dev-emulator-program.nrel-op.json
Successfully downloaded config with version 1 for Development environment (program) and data collection URL default
Dynamic labels download was successful for nrel-openpath-deploy-configs: dev-emulator-program
Running at 2023-11-20T17:05:28.739809+00:00 with args Namespace(plot_notebook='generic_metrics.ipynb', program='default', date=None) for range (<Arrow [2020-09-01T00:00:00+00:00]>, <Arrow [2023-11-01T00:00:00+00:00]>)
Running at 2023-11-20T17:05:28.779264+00:00 with params [Parameter('year', int), Parameter('month', int), Parameter('program', str, value='default'), Parameter('study_type', str, value='program'), Parameter('include_test_users', bool, value=False), Parameter('dynamic_labels', dict, value={'MODE': [{'value': 'walk', 'baseMode': 'WALKING', 'met_equivalent': 'WALKING', 'kgCo2PerKm': 0}, {'value': 'e-bike', 'baseMode': 'E_BIKE', 'met': {'ALL': {'range': [0, -1], 'mets': 4.9}}, 'kgCo2PerKm': 0.00728}, {'value': 'bike', 'baseMode': 'BICYCLING', 'met_equivalent': 'BICYCLING', 'kgCo2PerKm': 0}, {'value': 'bikeshare', 'baseMode': 'BICYCLING', 'met_equivalent': 'BICYCLING', 'kgCo2PerKm': 0}, {'value': 'scootershare', 'baseMode': 'E_SCOOTER', 'met_equivalent': 'IN_VEHICLE', 'kgCo2PerKm': 0.00894}, {'value': 'drove_alone', 'baseMode': 'CAR', 'met_equivalent': 'IN_VEHICLE', 'kgCo2PerKm': 0.22031}, {'value': 'shared_ride', 'baseMode': 'CAR', 'met_equivalent': 'IN_VEHICLE', 'kgCo2PerKm': 0.11015}, {'value': 'e_car_drove_alone', 'baseMode': 'E_CAR', 'met_equivalent': 'IN_VEHICLE', 'kgCo2PerKm': 0.08216}, {'value': 'e_car_shared_ride', 'baseMode': 'E_CAR', 'met_equivalent': 'IN_VEHICLE', 'kgCo2PerKm': 0.04108}, {'value': 'moped', 'baseMode': 'MOPED', 'met_equivalent': 'IN_VEHICLE', 'kgCo2PerKm': 0.05555}, {'value': 'taxi', 'baseMode': 'TAXI', 'met_equivalent': 'IN_VEHICLE', 'kgCo2PerKm': 0.30741}, {'value': 'bus', 'baseMode': 'BUS', 'met_equivalent': 'IN_VEHICLE', 'kgCo2PerKm': 0.20727}, {'value': 'train', 'baseMode': 'TRAIN', 'met_equivalent': 'IN_VEHICLE', 'kgCo2PerKm': 0.12256}, {'value': 'free_shuttle', 'baseMode': 'BUS', 'met_equivalent': 'IN_VEHICLE', 'kgCo2PerKm': 0.20727}, {'value': 'air', 'baseMode': 'AIR', 'met_equivalent': 'IN_VEHICLE', 'kgCo2PerKm': 0.09975}, {'value': 'not_a_trip', 'baseMode': 'UNKNOWN', 'met_equivalent': 'UNKNOWN', 'kgCo2PerKm': 0}, {'value': 'other', 'baseMode': 'OTHER', 'met_equivalent': 'UNKNOWN', 'kgCo2PerKm': 0}], 'PURPOSE': [{'value': 'home'}, {'value': 'work'}, {'value': 'at_work'}, {'value': 'school'}, {'value': 'transit_transfer'}, {'value': 'shopping'}, {'value': 'meal'}, {'value': 'pick_drop_person'}, {'value': 'pick_drop_item'}, {'value': 'personal_med'}, {'value': 'access_recreation'}, {'value': 'exercise'}, {'value': 'entertainment'}, {'value': 'religious'}, {'value': 'other'}], 'REPLACED_MODE': [{'value': 'no_travel'}, {'value': 'walk'}, {'value': 'bike'}, {'value': 'bikeshare'}, {'value': 'scootershare'}, {'value': 'drove_alone'}, {'value': 'shared_ride'}, {'value': 'e_car_drove_alone'}, {'value': 'e_car_shared_ride'}, {'value': 'taxi'}, {'value': 'bus'}, {'value': 'train'}, {'value': 'free_shuttle'}, {'value': 'other'}], 'translations': {'en': {'walk': 'Walk', 'e-bike': 'E-bike', 'bike': 'Regular Bike', 'bikeshare': 'Bikeshare', 'scootershare': 'Scooter share', 'drove_alone': 'Gas Car Drove Alone', 'shared_ride': 'Gas Car Shared Ride', 'e_car_drove_alone': 'E-Car Drove Alone', 'e_car_shared_ride': 'E-Car Shared Ride', 'moped': 'Moped', 'taxi': 'Taxi/Uber/Lyft', 'bus': 'Bus', 'train': 'Train', 'free_shuttle': 'Free Shuttle', 'air': 'Air', 'not_a_trip': 'Not a trip', 'no_travel': 'No travel', 'home': 'Home', 'work': 'To Work', 'at_work': 'At Work', 'school': 'School', 'transit_transfer': 'Transit transfer', 'shopping': 'Shopping', 'meal': 'Meal', 'pick_drop_person': 'Pick-up/ Drop off Person', 'pick_drop_item': 'Pick-up/ Drop off Item', 'personal_med': 'Personal/ Medical', 'access_recreation': 'Access Recreation', 'exercise': 'Recreation/ Exercise', 'entertainment': 'Entertainment/ Social', 'religious': 'Religious', 'other': 'Other'}, 'es': {'walk': 'Caminando', 'e-bike': 'e-bicicleta', 'bike': 'Bicicleta', 'bikeshare': 'Bicicleta compartida', 'scootershare': 'Motoneta compartida', 'drove_alone': 'Coche de Gas, Condujo solo', 'shared_ride': 'Coche de Gas, Condujo con otros', 'e_car_drove_alone': 'e-coche, Condujo solo', 'e_car_shared_ride': 'e-coche, Condujo con ontras', 'moped': 'Ciclomotor', 'taxi': 'Taxi/Uber/Lyft', 'bus': 'Autobús', 'train': 'Tren', 'free_shuttle': 'Colectivo gratuito', 'air': 'Avión', 'not_a_trip': 'No es un viaje', 'no_travel': 'No viajar', 'home': 'Inicio', 'work': 'Trabajo', 'at_work': 'En el trabajo', 'school': 'Escuela', 'transit_transfer': 'Transbordo', 'shopping': 'Compras', 'meal': 'Comida', 'pick_drop_person': 'Recoger/ Entregar Individuo', 'pick_drop_item': 'Recoger/ Entregar Objeto', 'personal_med': 'Personal/ Médico', 'access_recreation': 'Acceder a Recreación', 'exercise': 'Recreación/ Ejercicio', 'entertainment': 'Entretenimiento/ Social', 'religious': 'Religioso', 'other': 'Otros'}}})]

Results:

Overall Comparison:

Default Mapping Dynamic Config
Screenshot 2023-11-20 at 9 58 48 AM Screenshot 2023-11-20 at 10 21 41 AM

Individual Comparisons:

Number of Trips

Default Mapping Dynamic Config
image image

Number of commute Trips

Default Mapping Dynamic Config
image image

Trip count by purpose

Default Mapping Dynamic Config
image image

Trip count under 10 miles

Default Mapping Dynamic Config
image Screenshot 2023-11-20 at 1 10 29 PM

Trip miles by mode

Default Mapping Dynamic Config
image image

Average Trip Length

Default Mapping Dynamic Config
image image

Trip Frequency

Default Mapping Dynamic Config
image image

Trip Frequency (Weekday)

Default Mapping Dynamic Config
image image

Note: There is no mapping for "pilot_ebike" to "E-bike". Therefore, these mapping has been transformed into "Others" in "Number of Commute Trips" and "Trip count by Purpose" charts above.

Overall the charts looks identical.

UPDATE: There is some disparity in the charts depicted above, which has been explained below:
#91 (comment)

@iantei
Copy link
Contributor Author

iantei commented Nov 21, 2023

  • I want to see an explanation of the sensed notebooks in the Lao case

I am observing some errors with sensed notebook for Lao case.

Changes:

STUDY_CONFIG=`usaid-laos-ev`
Using vail dataset.

I am getting the following errors:

Error Details

/usr/src/app/saved-notebooks/bin/generate_plots.py:30: SyntaxWarning: "is not" with a literal. Did you mean "!="? if r.status_code is not 200: About to download config from https://raw.githubusercontent.com/e-mission/nrel-openpath-deploy-configs/main/configs/usaid-laos-ev.nrel-op.json Successfully downloaded config with version 1 for USAID-NREL Support for Electric Vehicle Readiness and data collection URL https://USAID-laos-EV-openpath.nrel.gov/api/ Running at 2023-11-21T00:26:03.065634+00:00 with args Namespace(plot_notebook='generic_metrics_sensed.ipynb', program='default', date=None) for range (, ) Running at 2023-11-21T00:26:03.105566+00:00 with params [Parameter('program', str, value='default'), Parameter('study_type', str, value='study'), Parameter('include_test_users', bool, value=True), Parameter('sensed_algo_prefix', str, value='cleaned')] Traceback (most recent call last): File "/usr/src/app/saved-notebooks/bin/generate_plots.py", line 84, in compute_for_date(None, None) File "/usr/src/app/saved-notebooks/bin/generate_plots.py", line 81, in compute_for_date nbclient.execute(new_nb) File "/root/miniconda-23.1.0/envs/emission/lib/python3.9/site-packages/nbclient/client.py", line 1305, in execute return NotebookClient(nb=nb, resources=resources, km=km, **kwargs).execute() File "/root/miniconda-23.1.0/envs/emission/lib/python3.9/site-packages/jupyter_core/utils/__init__.py", line 166, in wrapped return loop.run_until_complete(inner) File "/root/miniconda-23.1.0/envs/emission/lib/python3.9/asyncio/base_events.py", line 647, in run_until_complete return future.result() File "/root/miniconda-23.1.0/envs/emission/lib/python3.9/site-packages/nbclient/client.py", line 705, in async_execute await self.async_execute_cell( File "/root/miniconda-23.1.0/envs/emission/lib/python3.9/site-packages/nbclient/client.py", line 1058, in async_execute_cell await self._check_raise_for_error(cell, cell_index, exec_reply) File "/root/miniconda-23.1.0/envs/emission/lib/python3.9/site-packages/nbclient/client.py", line 914, in _check_raise_for_error raise CellExecutionError.from_cell_and_msg(cell, exec_reply_content) nbclient.exceptions.CellExecutionError: An error occurred while executing the following cell: ------------------ expanded_ct, file_suffix, quality_text, debug_df = scaffolding.load_viz_notebook_sensor_inference_data(year, month, program, include_test_users, sensed_algo_prefix) ------------------

----- stdout -----
Loaded all confirmed trips of length 57407
----- stdout -----
After filtering, found 57407 participant trips
----- stdout -----
Loaded expanded_ct with length 57407 for None


TypeError Traceback (most recent call last)
Cell In[3], line 1
----> 1 expanded_ct, file_suffix, quality_text, debug_df = scaffolding.load_viz_notebook_sensor_inference_data(year,
2 month,
3 program,
4 include_test_users,
5 sensed_algo_prefix)

File /usr/src/app/saved-notebooks/scaffolding.py:175, in load_viz_notebook_sensor_inference_data(year, month, program, include_test_users, sensed_algo_prefix)
173 print(f"Loaded expanded_ct with length {len(expanded_ct)} for {tq}")
174 if len(expanded_ct) > 0:
--> 175 expanded_ct["primary_mode_non_other"] = participant_ct_df.cleaned_section_summary.apply(lambda md: max(md["distance"], key=md["distance"].get))
176 expanded_ct.primary_mode_non_other.replace({"ON_FOOT": "WALKING"}, inplace=True)
177 valid_sensed_modes = ["WALKING", "BICYCLING", "IN_VEHICLE", "AIR_OR_HSR", "UNKNOWN"]

File ~/miniconda-23.1.0/envs/emission/lib/python3.9/site-packages/pandas/core/series.py:4771, in Series.apply(self, func, convert_dtype, args, **kwargs)
4661 def apply(
4662 self,
4663 func: AggFuncType,
(...)
4666 **kwargs,
4667 ) -> DataFrame | Series:
4668 """
4669 Invoke function on values of Series.
4670
(...)
4769 dtype: float64
4770 """
-> 4771 return SeriesApply(self, func, convert_dtype, args, kwargs).apply()

File ~/miniconda-23.1.0/envs/emission/lib/python3.9/site-packages/pandas/core/apply.py:1123, in SeriesApply.apply(self)
1120 return self.apply_str()
1122 # self.f is Callable
-> 1123 return self.apply_standard()

File ~/miniconda-23.1.0/envs/emission/lib/python3.9/site-packages/pandas/core/apply.py:1174, in SeriesApply.apply_standard(self)
1172 else:
1173 values = obj.astype(object)._values
-> 1174 mapped = lib.map_infer(
1175 values,
1176 f,
1177 convert=self.convert_dtype,
1178 )
1180 if len(mapped) and isinstance(mapped[0], ABCSeries):
1181 # GH#43986 Need to do list(mapped) in order to get treated as nested
1182 # See also GH#25959 regarding EA support
1183 return obj._constructor_expanddim(list(mapped), index=obj.index)

File ~/miniconda-23.1.0/envs/emission/lib/python3.9/site-packages/pandas/_libs/lib.pyx:2924, in pandas._libs.lib.map_infer()

File /usr/src/app/saved-notebooks/scaffolding.py:175, in load_viz_notebook_sensor_inference_data..(md)
173 print(f"Loaded expanded_ct with length {len(expanded_ct)} for {tq}")
174 if len(expanded_ct) > 0:
--> 175 expanded_ct["primary_mode_non_other"] = participant_ct_df.cleaned_section_summary.apply(lambda md: max(md["distance"], key=md["distance"].get))
176 expanded_ct.primary_mode_non_other.replace({"ON_FOOT": "WALKING"}, inplace=True)
177 valid_sensed_modes = ["WALKING", "BICYCLING", "IN_VEHICLE", "AIR_OR_HSR", "UNKNOWN"]

TypeError: 'float' object is not subscriptable

(emission) root@b4d4d64a5353:/usr/src/app/saved-notebooks#

Tried with the main repo, without my changes related with dynamic config. Still getting the same error.

#89 (comment)
Referring to the above Issue discussion, there are instances of NaN values in the data which is causing this issue.

The results are displayed below with some code changes on scaffolding.py to debug:

Code changes and Results

Code changes:

    participant_ct_df = all_ct[all_ct.user_id.isin(participant_list)]

    if participant_ct_df.isna().values.any():
        print("The DataFrame contains missing values (NaN).")
    else:
        print("The DataFrame does not contain any missing values.")
    print(participant_ct_df.inferred_section_summary[participant_ct_df.inferred_section_summary.isna()])
    print("After filtering, found %s participant trips " % len(participant_ct_df))

Results:

The DataFrame contains missing values (NaN).
0        NaN
1        NaN
2        NaN
3        NaN
4        NaN
        ... 
57396    NaN
57397    NaN
57398    NaN
57399    NaN
57400    NaN
Name: inferred_section_summary, Length: 57401, dtype: object
After filtering, found 57407 participant trips 

@iantei
Copy link
Contributor Author

iantei commented Nov 21, 2023

@iantei looking through my prior comments to make sure that they are all resolved, I ran into this comment.

Similarly, notebook execution for sensed notebook failed since we have some NaN values in the dataset used - as justified above.

As justified where? The sensed notebooks should not be config specific since we don't use user labels in them.

The conversation was documented in Issue section, #89 (comment)
For my case where I am using vail dataset, I am encountering issue which I have enlisted above.

…xample-program-label-options for print_CO2_emission_calculations().
@shankari
Copy link
Contributor

shankari commented Nov 22, 2023

Overall the charts looks identical.

Not really. I still see differences in:

  1. Trip count by purpose

I see your comment around

Note: There is no mapping for "pilot_ebike" to "E-bike". Therefore, these mapping has been transformed into "Others" in "Number of Commute Trips" and "Trip count by Purpose" charts above.

But "Trip count by Purpose" does not display modes, so a difference in the e-bike label should not make any difference. And there is no "other" in the purpose charts anyway. It does cause the large "other" in "Number of trips", so I have not flagged that.

  1. Trip count under 10 miles is comparing modes to purpose
  2. Trip miles by mode
  3. Average trip length

Also, "generic_metrics" as an example (note the e.g. before it). I would like to see at least a few metrics from the mode specific and energy/emission notebooks.

I still want to see all the outputs (e.g. generic metrics) one last time to make sure that there are no regressions

@iantei
Copy link
Contributor Author

iantei commented Nov 24, 2023

Comparison of mode specific metrics outputs:

Executed the notebook "Mode Specific Metrics" for both the STUDY_CONFIG for comparison between default mapping and dynamic config.

Steps executed:
A. For Default Mapping (STUDY_CONFIG=stage-program):

(emission) root@1d89306b8d05:/usr/src/app/saved-notebooks# PYTHONPATH=.. python bin/update_mappings.py mapping_dictionaries.ipynb
(emission) root@1d89306b8d05:/usr/src/app/saved-notebooks# PYTHONPATH=.. python bin/generate_plots.py mode_specific_metrics.ipynb default
/usr/src/app/saved-notebooks/bin/generate_plots.py:30: SyntaxWarning: "is not" with a literal. Did you mean "!="?
  if r.status_code is not 200:
About to download config from https://raw.githubusercontent.com/e-mission/nrel-openpath-deploy-configs/main/configs/stage-program.nrel-op.json
Successfully downloaded config with version 1 for Staging environment for testing programs only and data collection URL https://openpath-stage.nrel.gov/api/
label_options is unavailable for the dynamic_config in stage-program
Running at 2023-11-24T19:16:46.712768+00:00 with args Namespace(plot_notebook='mode_specific_metrics.ipynb', program='default', date=None) for range (<Arrow [2020-09-01T00:00:00+00:00]>, <Arrow [2023-11-01T00:00:00+00:00]>)
Running at 2023-11-24T19:16:46.748639+00:00 with params [Parameter('year', int), Parameter('month', int), Parameter('program', str, value='default'), Parameter('study_type', str, value='program'), Parameter('mode_of_interest', str, value='e-bike'), Parameter('include_test_users', bool, value=True), Parameter('dynamic_labels', dict, value={})]
Running at 2023-11-24T19:17:32.189017+00:00 with params [Parameter('year', int, value=2020), Parameter('month', int, value=9), Parameter('program', str, value='default'), Parameter('study_type', str, value='program'), Parameter('mode_of_interest', str, value='e-bike'), Parameter('include_test_users', bool, value=True), Parameter('dynamic_labels', dict, value={})]
Running at 2023-11-24T19:17:38.020350+00:00 with params [Parameter('year', int, value=2020), Parameter('month', int, value=10), Parameter('program', str, value='default'), Parameter('study_type', str, value='program'), Parameter('mode_of_interest', str, value='e-bike'), Parameter('include_test_users', bool, value=True), Parameter('dynamic_labels', dict, value={})]
Running at 2023-11-24T19:17:43.716868+00:00 with params [Parameter('year', int, value=2020), Parameter('month', int, value=11), Parameter('program', str, value='default'), Parameter('study_type', str, value='program'), Parameter('mode_of_interest', str, value='e-bike'), Parameter('include_test_users', bool, value=True), Parameter('dynamic_labels', dict, value={})]
Running at 2023-11-24T19:17:50.061952+00:00 with params [Parameter('year', int, value=2020), Parameter('month', int, value=12), Parameter('program', str, value='default'), Parameter('study_type', str, value='program'), Parameter('mode_of_interest', str, value='e-bike'), Parameter('include_test_users', bool, value=True), Parameter('dynamic_labels', dict, value={})]

B. For Dynamic Config (STUDY_CONFIG=dev-emulator-program):

(emission) root@f1a86a1f3abd:/usr/src/app/saved-notebooks# PYTHONPATH=.. python bin/update_mappings.py mapping_dictionaries.ipynb
(emission) root@f1a86a1f3abd:/usr/src/app/saved-notebooks# PYTHONPATH=.. python bin/generate_plots.py mode_specific_metrics.ipynb default
/usr/src/app/saved-notebooks/bin/generate_plots.py:30: SyntaxWarning: "is not" with a literal. Did you mean "!="?
  if r.status_code is not 200:
About to download config from https://raw.githubusercontent.com/e-mission/nrel-openpath-deploy-configs/main/configs/dev-emulator-program.nrel-op.json
Successfully downloaded config with version 1 for Development environment (program) and data collection URL default
Dynamic labels download was successful for nrel-openpath-deploy-configs: dev-emulator-program
Running at 2023-11-24T18:33:55.368650+00:00 with args Namespace(plot_notebook='mode_specific_metrics.ipynb', program='default', date=None) for range (<Arrow [2020-09-01T00:00:00+00:00]>, <Arrow [2023-11-01T00:00:00+00:00]>)
Running at 2023-11-24T18:33:55.404501+00:00 with params [Parameter('year', int), Parameter('month', int), Parameter('program', str, value='default'), Parameter('study_type', str, value='program'), Parameter('mode_of_interest', str, value='e-bike'), Parameter('include_test_users', bool, value=False), Parameter('dynamic_labels', dict, value={'MODE': [{'value': 'walk', 'baseMode': 'WALKING', 'met_equivalent': 'WALKING', 'kgCo2PerKm': 0}, {'value': 'e-bike', 'baseMode': 'E_BIKE', 'met': {'ALL': {'range': [0, -1], 'mets': 4.9}}, 'kgCo2PerKm': 0.00728}, {'value': 'bike', 'baseMode': 'BICYCLING', 'met_equivalent': 'BICYCLING', 'kgCo2PerKm': 0}, {'value': 'bikeshare', 'baseMode': 'BICYCLING', 'met_equivalent': 'BICYCLING', 'kgCo2PerKm': 0}, {'value': 'scootershare', 'baseMode': 'E_SCOOTER', 'met_equivalent': 'IN_VEHICLE', 'kgCo2PerKm': 0.00894}, {'value': 'drove_alone', 'baseMode': 'CAR', 'met_equivalent': 'IN_VEHICLE', 'kgCo2PerKm': 0.22031}, {'value': 'shared_ride', 'baseMode': 'CAR', 'met_equivalent': 'IN_VEHICLE', 'kgCo2PerKm': 0.11015}, {'value': 'e_car_drove_alone', 'baseMode': 'E_CAR', 'met_equivalent': 'IN_VEHICLE', 'kgCo2PerKm': 0.08216}, {'value': 'e_car_shared_ride', 'baseMode': 'E_CAR', 'met_equivalent': 'IN_VEHICLE', 'kgCo2PerKm': 0.04108}, {'value': 'moped', 'baseMode': 'MOPED', 'met_equivalent': 'IN_VEHICLE', 'kgCo2PerKm': 0.05555}, {'value': 'taxi', 'baseMode': 'TAXI', 'met_equivalent': 'IN_VEHICLE', 'kgCo2PerKm': 0.30741}, {'value': 'bus', 'baseMode': 'BUS', 'met_equivalent': 'IN_VEHICLE', 'kgCo2PerKm': 0.20727}, {'value': 'train', 'baseMode': 'TRAIN', 'met_equivalent': 'IN_VEHICLE', 'kgCo2PerKm': 0.12256}, {'value': 'free_shuttle', 'baseMode': 'BUS', 'met_equivalent': 'IN_VEHICLE', 'kgCo2PerKm': 0.20727}, {'value': 'air', 'baseMode': 'AIR', 'met_equivalent': 'IN_VEHICLE', 'kgCo2PerKm': 0.09975}, {'value': 'not_a_trip', 'baseMode': 'UNKNOWN', 'met_equivalent': 'UNKNOWN', 'kgCo2PerKm': 0}, {'value': 'other', 'baseMode': 'OTHER', 'met_equivalent': 'UNKNOWN', 'kgCo2PerKm': 0}], 'PURPOSE': [{'value': 'home'}, {'value': 'work'}, {'value': 'at_work'}, {'value': 'school'}, {'value': 'transit_transfer'}, {'value': 'shopping'}, {'value': 'meal'}, {'value': 'pick_drop_person'}, {'value': 'pick_drop_item'}, {'value': 'personal_med'}, {'value': 'access_recreation'}, {'value': 'exercise'}, {'value': 'entertainment'}, {'value': 'religious'}, {'value': 'other'}], 'REPLACED_MODE': [{'value': 'no_travel'}, {'value': 'walk'}, {'value': 'bike'}, {'value': 'bikeshare'}, {'value': 'scootershare'}, {'value': 'drove_alone'}, {'value': 'shared_ride'}, {'value': 'e_car_drove_alone'}, {'value': 'e_car_shared_ride'}, {'value': 'taxi'}, {'value': 'bus'}, {'value': 'train'}, {'value': 'free_shuttle'}, {'value': 'other'}], 'translations': {'en': {'walk': 'Walk', 'e-bike': 'E-bike', 'bike': 'Regular Bike', 'bikeshare': 'Bikeshare', 'scootershare': 'Scooter share', 'drove_alone': 'Gas Car Drove Alone', 'shared_ride': 'Gas Car Shared Ride', 'e_car_drove_alone': 'E-Car Drove Alone', 'e_car_shared_ride': 'E-Car Shared Ride', 'moped': 'Moped', 'taxi': 'Taxi/Uber/Lyft', 'bus': 'Bus', 'train': 'Train', 'free_shuttle': 'Free Shuttle', 'air': 'Air', 'not_a_trip': 'Not a trip', 'no_travel': 'No travel', 'home': 'Home', 'work': 'To Work', 'at_work': 'At Work', 'school': 'School', 'transit_transfer': 'Transit transfer', 'shopping': 'Shopping', 'meal': 'Meal', 'pick_drop_person': 'Pick-up/ Drop off Person', 'pick_drop_item': 'Pick-up/ Drop off Item', 'personal_med': 'Personal/ Medical', 'access_recreation': 'Access Recreation', 'exercise': 'Recreation/ Exercise', 'entertainment': 'Entertainment/ Social', 'religious': 'Religious', 'other': 'Other'}, 'es': {'walk': 'Caminando', 'e-bike': 'e-bicicleta', 'bike': 'Bicicleta', 'bikeshare': 'Bicicleta compartida', 'scootershare': 'Motoneta compartida', 'drove_alone': 'Coche de Gas, Condujo solo', 'shared_ride': 'Coche de Gas, Condujo con otros', 'e_car_drove_alone': 'e-coche, Condujo solo', 'e_car_shared_ride': 'e-coche, Condujo con ontras', 'moped': 'Ciclomotor', 'taxi': 'Taxi/Uber/Lyft', 'bus': 'Autobús', 'train': 'Tren', 'free_shuttle': 'Colectivo gratuito', 'air': 'Avión', 'not_a_trip': 'No es un viaje', 'no_travel': 'No viajar', 'home': 'Inicio', 'work': 'Trabajo', 'at_work': 'En el trabajo', 'school': 'Escuela', 'transit_transfer': 'Transbordo', 'shopping': 'Compras', 'meal': 'Comida', 'pick_drop_person': 'Recoger/ Entregar Individuo', 'pick_drop_item': 'Recoger/ Entregar Objeto', 'personal_med': 'Personal/ Médico', 'access_recreation': 'Acceder a Recreación', 'exercise': 'Recreación/ Ejercicio', 'entertainment': 'Entretenimiento/ Social', 'religious': 'Religioso', 'other': 'Otros'}}})]

Results:

Overall Comparison

Default Mapping Dynamic Config
Screenshot 2023-11-24 at 12 36 06 PM Screenshot 2023-11-24 at 12 08 17 PM

'e-bike' specific trip count by purpose All data Default

Default Mapping Dynamic Config
Screenshot 2023-11-24 at 12 36 15 PM Screenshot 2023-11-24 at 12 08 27 PM

'e-bike' specific trip miles by replaced mode All data Default

Default Mapping Dynamic Config
Screenshot 2023-11-24 at 12 36 22 PM Screenshot 2023-11-24 at 12 08 35 PM

Average Miles for each replaced mode (w/Other) All data Default

Default Mapping Dynamic Config
Screenshot 2023-11-24 at 12 36 27 PM Screenshot 2023-11-24 at 12 08 41 PM

'e-bike' specific Trip frequency All data

Default Mapping Dynamic Config
Screenshot 2023-11-24 at 12 36 33 PM Screenshot 2023-11-24 at 12 08 47 PM

'e-bike' specific Trips per weekday

Default Mapping Dynamic Config
Screenshot 2023-11-24 at 12 36 40 PM Screenshot 2023-11-24 at 12 08 53 PM

All the charts in both Default Mapping and Dynamic Config for Mode Specific Metrics looks identical.

@iantei
Copy link
Contributor Author

iantei commented Nov 24, 2023

Comparison of energy calculations notebook charts

Executed the notebook "Energy Calculations" for both the STUDY_CONFIG for comparison between default mapping and dynamic config.

Steps executed:
A. For Default Mapping (STUDY_CONFIG=stage-program):

(emission) root@1d89306b8d05:/usr/src/app/saved-notebooks# PYTHONPATH=.. python bin/generate_plots.py energy_calculations.ipynb default
/usr/src/app/saved-notebooks/bin/generate_plots.py:30: SyntaxWarning: "is not" with a literal. Did you mean "!="?
  if r.status_code is not 200:
About to download config from https://raw.githubusercontent.com/e-mission/nrel-openpath-deploy-configs/main/configs/stage-program.nrel-op.json
Successfully downloaded config with version 1 for Staging environment for testing programs only and data collection URL https://openpath-stage.nrel.gov/api/
label_options is unavailable for the dynamic_config in stage-program
Running at 2023-11-24T19:24:31.229825+00:00 with args Namespace(plot_notebook='energy_calculations.ipynb', program='default', date=None) for range (<Arrow [2020-09-01T00:00:00+00:00]>, <Arrow [2023-11-01T00:00:00+00:00]>)
Running at 2023-11-24T19:24:31.278778+00:00 with params [Parameter('year', int), Parameter('month', int), Parameter('program', str, value='default'), Parameter('study_type', str, value='program'), Parameter('mode_of_interest', str, value='e-bike'), Parameter('include_test_users', bool, value=True), Parameter('is_debug_mode', bool, value=False), Parameter('dynamic_labels', dict, value={})]
Running at 2023-11-24T19:24:46.321639+00:00 with params [Parameter('year', int, value=2020), Parameter('month', int, value=9), Parameter('program', str, value='default'), Parameter('study_type', str, value='program'), Parameter('mode_of_interest', str, value='e-bike'), Parameter('include_test_users', bool, value=True), Parameter('is_debug_mode', bool, value=False), Parameter('dynamic_labels', dict, value={})]
Running at 2023-11-24T19:24:50.173058+00:00 with params [Parameter('year', int, value=2020), Parameter('month', int, value=10), Parameter('program', str, value='default'), Parameter('study_type', str, value='program'), Parameter('mode_of_interest', str, value='e-bike'), Parameter('include_test_users', bool, value=True), Parameter('is_debug_mode', bool, value=False), Parameter('dynamic_labels', dict, value={})]
Running at 2023-11-24T19:24:53.866005+00:00 with params [Parameter('year', int, value=2020), Parameter('month', int, value=11), Parameter('program', str, value='default'), Parameter('study_type', str, value='program'), Parameter('mode_of_interest', str, value='e-bike'), Parameter('include_test_users', bool, value=True), Parameter('is_debug_mode', bool, value=False), Parameter('dynamic_labels', dict, value={})]
Running at 2023-11-24T19:24:57.622034+00:00 with params [Parameter('year', int, value=2020), Parameter('month', int, value=12), Parameter('program', str, value='default'), Parameter('study_type', str, value='program'), Parameter('mode_of_interest', str, value='e-bike'), Parameter('include_test_users', bool, value=True), Parameter('is_debug_mode', bool, value=False), Parameter('dynamic_labels', dict, value={})]

B. For Dynamic Config (STUDY_CONFIG=dev-emulator-program):

(emission) root@f1a86a1f3abd:/usr/src/app/saved-notebooks# PYTHONPATH=.. python bin/generate_plots.py energy_calculations.ipynb default
/usr/src/app/saved-notebooks/bin/generate_plots.py:30: SyntaxWarning: "is not" with a literal. Did you mean "!="?
  if r.status_code is not 200:
About to download config from https://raw.githubusercontent.com/e-mission/nrel-openpath-deploy-configs/main/configs/dev-emulator-program.nrel-op.json
Successfully downloaded config with version 1 for Development environment (program) and data collection URL default
Dynamic labels download was successful for nrel-openpath-deploy-configs: dev-emulator-program
Running at 2023-11-24T18:41:28.356249+00:00 with args Namespace(plot_notebook='energy_calculations.ipynb', program='default', date=None) for range (<Arrow [2020-09-01T00:00:00+00:00]>, <Arrow [2023-11-01T00:00:00+00:00]>)
Running at 2023-11-24T18:41:28.390824+00:00 with params [Parameter('year', int), Parameter('month', int), Parameter('program', str, value='default'), Parameter('study_type', str, value='program'), Parameter('mode_of_interest', str, value='e-bike'), Parameter('include_test_users', bool, value=False), Parameter('is_debug_mode', bool, value=False), Parameter('dynamic_labels', dict, value={'MODE': [{'value': 'walk', 'baseMode': 'WALKING', 'met_equivalent': 'WALKING', 'kgCo2PerKm': 0}, {'value': 'e-bike', 'baseMode': 'E_BIKE', 'met': {'ALL': {'range': [0, -1], 'mets': 4.9}}, 'kgCo2PerKm': 0.00728}, {'value': 'bike', 'baseMode': 'BICYCLING', 'met_equivalent': 'BICYCLING', 'kgCo2PerKm': 0}, {'value': 'bikeshare', 'baseMode': 'BICYCLING', 'met_equivalent': 'BICYCLING', 'kgCo2PerKm': 0}, {'value': 'scootershare', 'baseMode': 'E_SCOOTER', 'met_equivalent': 'IN_VEHICLE', 'kgCo2PerKm': 0.00894}, {'value': 'drove_alone', 'baseMode': 'CAR', 'met_equivalent': 'IN_VEHICLE', 'kgCo2PerKm': 0.22031}, {'value': 'shared_ride', 'baseMode': 'CAR', 'met_equivalent': 'IN_VEHICLE', 'kgCo2PerKm': 0.11015}, {'value': 'e_car_drove_alone', 'baseMode': 'E_CAR', 'met_equivalent': 'IN_VEHICLE', 'kgCo2PerKm': 0.08216}, {'value': 'e_car_shared_ride', 'baseMode': 'E_CAR', 'met_equivalent': 'IN_VEHICLE', 'kgCo2PerKm': 0.04108}, {'value': 'moped', 'baseMode': 'MOPED', 'met_equivalent': 'IN_VEHICLE', 'kgCo2PerKm': 0.05555}, {'value': 'taxi', 'baseMode': 'TAXI', 'met_equivalent': 'IN_VEHICLE', 'kgCo2PerKm': 0.30741}, {'value': 'bus', 'baseMode': 'BUS', 'met_equivalent': 'IN_VEHICLE', 'kgCo2PerKm': 0.20727}, {'value': 'train', 'baseMode': 'TRAIN', 'met_equivalent': 'IN_VEHICLE', 'kgCo2PerKm': 0.12256}, {'value': 'free_shuttle', 'baseMode': 'BUS', 'met_equivalent': 'IN_VEHICLE', 'kgCo2PerKm': 0.20727}, {'value': 'air', 'baseMode': 'AIR', 'met_equivalent': 'IN_VEHICLE', 'kgCo2PerKm': 0.09975}, {'value': 'not_a_trip', 'baseMode': 'UNKNOWN', 'met_equivalent': 'UNKNOWN', 'kgCo2PerKm': 0}, {'value': 'other', 'baseMode': 'OTHER', 'met_equivalent': 'UNKNOWN', 'kgCo2PerKm': 0}], 'PURPOSE': [{'value': 'home'}, {'value': 'work'}, {'value': 'at_work'}, {'value': 'school'}, {'value': 'transit_transfer'}, {'value': 'shopping'}, {'value': 'meal'}, {'value': 'pick_drop_person'}, {'value': 'pick_drop_item'}, {'value': 'personal_med'}, {'value': 'access_recreation'}, {'value': 'exercise'}, {'value': 'entertainment'}, {'value': 'religious'}, {'value': 'other'}], 'REPLACED_MODE': [{'value': 'no_travel'}, {'value': 'walk'}, {'value': 'bike'}, {'value': 'bikeshare'}, {'value': 'scootershare'}, {'value': 'drove_alone'}, {'value': 'shared_ride'}, {'value': 'e_car_drove_alone'}, {'value': 'e_car_shared_ride'}, {'value': 'taxi'}, {'value': 'bus'}, {'value': 'train'}, {'value': 'free_shuttle'}, {'value': 'other'}], 'translations': {'en': {'walk': 'Walk', 'e-bike': 'E-bike', 'bike': 'Regular Bike', 'bikeshare': 'Bikeshare', 'scootershare': 'Scooter share', 'drove_alone': 'Gas Car Drove Alone', 'shared_ride': 'Gas Car Shared Ride', 'e_car_drove_alone': 'E-Car Drove Alone', 'e_car_shared_ride': 'E-Car Shared Ride', 'moped': 'Moped', 'taxi': 'Taxi/Uber/Lyft', 'bus': 'Bus', 'train': 'Train', 'free_shuttle': 'Free Shuttle', 'air': 'Air', 'not_a_trip': 'Not a trip', 'no_travel': 'No travel', 'home': 'Home', 'work': 'To Work', 'at_work': 'At Work', 'school': 'School', 'transit_transfer': 'Transit transfer', 'shopping': 'Shopping', 'meal': 'Meal', 'pick_drop_person': 'Pick-up/ Drop off Person', 'pick_drop_item': 'Pick-up/ Drop off Item', 'personal_med': 'Personal/ Medical', 'access_recreation': 'Access Recreation', 'exercise': 'Recreation/ Exercise', 'entertainment': 'Entertainment/ Social', 'religious': 'Religious', 'other': 'Other'}, 'es': {'walk': 'Caminando', 'e-bike': 'e-bicicleta', 'bike': 'Bicicleta', 'bikeshare': 'Bicicleta compartida', 'scootershare': 'Motoneta compartida', 'drove_alone': 'Coche de Gas, Condujo solo', 'shared_ride': 'Coche de Gas, Condujo con otros', 'e_car_drove_alone': 'e-coche, Condujo solo', 'e_car_shared_ride': 'e-coche, Condujo con ontras', 'moped': 'Ciclomotor', 'taxi': 'Taxi/Uber/Lyft', 'bus': 'Autobús', 'train': 'Tren', 'free_shuttle': 'Colectivo gratuito', 'air': 'Avión', 'not_a_trip': 'No es un viaje', 'no_travel': 'No viajar', 'home': 'Inicio', 'work': 'Trabajo', 'at_work': 'En el trabajo', 'school': 'Escuela', 'transit_transfer': 'Transbordo', 'shopping': 'Compras', 'meal': 'Comida', 'pick_drop_person': 'Recoger/ Entregar Individuo', 'pick_drop_item': 'Recoger/ Entregar Objeto', 'personal_med': 'Personal/ Médico', 'access_recreation': 'Acceder a Recreación', 'exercise': 'Recreación/ Ejercicio', 'entertainment': 'Entretenimiento/ Social', 'religious': 'Religioso', 'other': 'Otros'}}})]

Results:

Sketch of 'e-bike' specific emission Impact All data Default

Default Mapping Dynamic Config
Screenshot 2023-11-24 at 12 37 30 PM Screenshot 2023-11-24 at 12 13 40 PM

The computation for Default Mapping is based on auxiliary_files/energy_intensity while the computation for Dynamic Config is based on https://github.com/e-mission/nrel-openpath-deploy-configs/blob/main/label_options/example-program-label-options.json.

The difference in these emissions have been documented here in the Issue section.
#89 (comment)

The explanation for this is documented above.
#91 (comment)

@iantei
Copy link
Contributor Author

iantei commented Nov 25, 2023

Response for the above comments are provided below [Generic Metrics Notebook Charts Difference Explaination]

Overall the charts looks identical.

Not really. I still see differences in:

  1. Trip count by purpose

I see your comment around

Note: There is no mapping for "pilot_ebike" to "E-bike". Therefore, these mapping has been transformed into "Others" in "Number of Commute Trips" and "Trip count by Purpose" charts above.

But "Trip count by Purpose" does not display modes, so a difference in the e-bike label should not make any difference. And there is no "other" in the purpose charts anyway. It does cause the large "other" in "Number of trips", so I have not flagged that.

For Trip count by Purpose, we have the following labels (identical in both Default Mapping and Dynamic Config) with the numeric representation of each labels

Purpose Number
Home 4458
Recreation/ Exercise 2575
Work 4655
Entertainment/ Social 1306
Shopping 2156
Personal/ Medical 947
Meal 847

All these labels are available on both Default Mapping and Dynamic Config. Whilst in the Dynamic Config, few more labels are introduced as. "At Work", "Other", "Pickup / Drop off item", "Pickup / Drop off Person". These are equivalent to the "Other" label present in the default mapping.

Further clarification showing comparison between two chart's label and it's value representation.

Default Mapping Dynamic Config
Screenshot 2023-11-28 at 3 15 45 PM Screenshot 2023-11-28 at 3 17 07 PM

As mentioned above, "Home", "Recreation/ Exercise", "Work" - "To Work", "Entertainment/Social", "Shopping", "Personal/Medical" and "Meal" have same corresponding values.

Lets understand the 'Other' label in the pie-chart.

For Default:
Other : 4360(Other) + 431 (not_a_trip) + 388 (School) + Pick-up/Drop off (163) + Religious (154) + Transit Transfer (114)  = 5610

For Dynamic Config:
Other: 1649 (Other) + 388 (School) + Access Recreation (211) + Religious (154) + Transit Transfer (114) = 2516.

For Dynamic Config, we have distinct new labels represented on the chart as: Pick-up/ Drop off Person and Pick-up/ Drop off Item

  1. Trip count under 10 miles is comparing modes to purpose

I understand this comment is in reference with "Mode choice for Trips under 10 miles". There was a wrong comparison in the charts above, which I have already updated. Now, explanation for the different in representation of the charts.

Code snippet to show we're using Mode_confirm.

    labels_d10 = expanded_ct.loc[(expanded_ct['distance_miles'] <= 10)].Mode_confirm.value_counts(dropna=True).keys().tolist()
    values_d10 = expanded_ct.loc[(expanded_ct['distance_miles'] <= 10)].Mode_confirm.value_counts(dropna=True).tolist()

The difference in representation is because of the following reasons.
For Default Mapping we have the following mapping (as in mode_labels.csv ):

pilot_ebike,pilot_ebike,E-bike
e-bike,e-bike,E-bike

while with the Dynamic Config, we just have (as in example_program_label_option)

{"value":"e-bike", "baseMode":"E_BIKE", "met": {"ALL": {"range": [0, -1], "mets": 4.9}}, "kgCo2PerKm": 0.00728},
...
  "translations": {
    "en": {
      "walk": "Walk",
      "e-bike": "E-bike",
      ...
}

in the reference example of config which I used to test. https://github.com/e-mission/nrel-openpath-deploy-configs/blob/main/label_options/example-program-label-options.json

Comparing labels and their corresponding values:

Default Mapping Dynamic Config
Default_under10miles Dynamic_under10miles

All the labels have same values except Other and E-bike.
Now, let us evaluate the sum of these labels.
For Default: 3853 (E-bike) + 180 (Other) = 4033
For Dynamic: 25 (E-bike) + 4008 (Other) = 4033
Sum of Other and E-bike labels in both of these cases give the total value of 4033.

This shows that because of the change in label for E-bike in the example config used for testing program_label_options, there is difference in representation. But the representation in both the charts are correct.

  1. Trip miles by mode

The addition of Air mode has resulted in this difference of representation.

Comparison of Average (miles) for Mode_confirm between Default Mapping vs Dynamic Config

Default Mapping Dynamic Config
DefaultMapping_TripMilesByMode DynamicConfig_TripMilesByMode

The Y-axis represents of Average(miles), with the Mode_confirm represented on the X-axis. As we can see from above, we have difference in Other and E-Bike. We didn't have Air label in Default Mapping which is available for the example Dynamic Config. Because of the no mapping translation from pilot_ebike to E-Bike, the average miles by E-Bike commute has dropped and added into Other label.
Therefore, there is a difference in representation of these values in the chart.

Re-checking, the total miles in both of these charts. Adding up the total Total (miles) in both of these gives same value as 142600.44744852.

  1. Average trip length

The representation includes Air mode, which has huge number of trip length. Therefore, the other modes are not represented as previously.

Also, "generic_metrics" as an example (note the e.g. before it). I would like to see at least a few metrics from the mode specific and energy/emission notebooks.

I still want to see all the outputs (e.g. generic metrics) one last time to make sure that there are no regressions

Updated the comparison charts between mode specific and energy emissions notebooks above.

@shankari
Copy link
Contributor

shankari commented Dec 3, 2023

I am not going to hold up the merge for this, but in the future, please test on a dataset with > 1 user and > 25 confirmed e-bike trips.

#91 (comment)
and
#91 (comment)

I am not sure we would have caught all the errors earlier with such a small dataset.
Please update the outputs into this PR after the merge with results from the larger dataset.

Please also note that I am squash-merging here, so please account for that while pulling post-merge.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants