From c3f88a6568e8a19ad7855735f11041a28bd8d8f9 Mon Sep 17 00:00:00 2001 From: Abby Wheelis <54848919+Abby-Wheelis@users.noreply.github.com> Date: Tue, 10 Oct 2023 14:35:20 -0600 Subject: [PATCH 1/4] instructions for loading data into the emulator after recent struggles remembering how to carry out the different variations of these tasks, documenting them here for future reference - load data from the sample files in the server - load data form gz files - work with a deployment config on the local server - make up a valid opcode --- docs/dev/front/loading_data_for_emulator.md | 46 +++++++++++++++++++++ 1 file changed, 46 insertions(+) create mode 100644 docs/dev/front/loading_data_for_emulator.md diff --git a/docs/dev/front/loading_data_for_emulator.md b/docs/dev/front/loading_data_for_emulator.md new file mode 100644 index 0000000..810ebca --- /dev/null +++ b/docs/dev/front/loading_data_for_emulator.md @@ -0,0 +1,46 @@ +# How to Load Data into Local Server for Use with devapp + +In order to fully test the functionality of the devapp, you will usually need to create an opcode with data. You **can** login to the devapp with a "real" opcode (ie the one you are logged into on a physical phone), which is useful for debugging. For more general development purposes, however, it is often more useful to work with sample data, loaded into an opcode of your choice. There are a few useful routes provided by the server for accomplishing this. + +## Load Test Data from Server + +You can find sample data in `e-mission-server/emission/tests/data/real_examples`. +Then, you can load the data into an opcode with the following two scripts. Make sure you have activated the emission environment first. + +```./e-mission-py.bash bin/debug/load_timeline_for_day_and_user.py emission/tests/data/real_examples/[file of your choice] nrelop_[complete valid opcode]``` + +```./e-mission-py.bash bin/debug/intake_single_user.py -e nrelop_[same valid opcode]``` + +After these two commands run sucessfully, you should be able to run the server and login with the opcode you created in the devapp. + +## Load Data from Local File + +Sometimes, you will have specific data that you want to load into the emulator. Once you have the two g-zipped files in a local directory, you can load the data into an opcode: + +`./e-mission-py bin/debug/load_multi_timeline_for_range.py [path/to/files/with/prefix` -- this path might look something like `/Users/name/Downloads/my_test_data/test_data` where `my_test_data` is a folder containing the two g-zipped files that each start with `test_data` + +The data will automatically create a "user email" for this opcode, by default it will be "user-0". This can be updated with: `edb.get_uuid_db().update_one({"user_email": "user-0"}, {"$set": {"user_email": "nrelop_usaid-laos-ev_default_test-1"}})` + +Then, process the data by running the pipeline. + +`./e-mission-py.bash bin/debug/intake_single_user.py -e [user-0 or updated]` + +If you updated to a valid opcode, you can now start the server and log into the devapp with that opcode + +## Not on Working with Deployment Configs + +There are development configs designed to be used with the local server. They include `dev-emulator-program`, `dev-emulator-study`, and `dev-emulator-timeuse`. + +Sometimes, it is useful to load data into a deployment config for testing, demo, or debugging purposes. If you want to use the local server and a deployment config, you need to take a few extra steps. +1. Fork `nrel-openpath-deploy-configs` +2. Modify the config you want to work on, making sure that there is no `server` defined (if no server is defined, the app will connect to the local server, as desired). +3. Update the downloadURL in `dynamicConfig` for the version of `e-mission-phone` you are running to `https://raw.githubusercontent.com/[MY-REPO]/nrel-openpath-deploy-configs/[MY-BRANCH]/configs/${label}.nrel-op.json` +4. Now you can use a development config with the local server + +## Notes on making up a valid opcode +- opcodes must begin with `nrelop` +- opcodes must contain the name of a config (ie `dev-emulator-study` or `nrel-commute`) +- some configs require a subgroup (ie `default` or `test`) +- the endings to opcodes can be anything, it is most useful to choose something memorable (ie `myTestData` or `test2`) + +All together the structure should be `nrelop_configName_subgroup_ending` From 502c00f1d2ce5d2abc474647b970c991aae2765a Mon Sep 17 00:00:00 2001 From: Abby Wheelis <54848919+Abby-Wheelis@users.noreply.github.com> Date: Tue, 10 Oct 2023 15:49:58 -0600 Subject: [PATCH 2/4] typo corrections --- docs/dev/front/loading_data_for_emulator.md | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/docs/dev/front/loading_data_for_emulator.md b/docs/dev/front/loading_data_for_emulator.md index 810ebca..6a1cc61 100644 --- a/docs/dev/front/loading_data_for_emulator.md +++ b/docs/dev/front/loading_data_for_emulator.md @@ -17,7 +17,7 @@ After these two commands run sucessfully, you should be able to run the server a Sometimes, you will have specific data that you want to load into the emulator. Once you have the two g-zipped files in a local directory, you can load the data into an opcode: -`./e-mission-py bin/debug/load_multi_timeline_for_range.py [path/to/files/with/prefix` -- this path might look something like `/Users/name/Downloads/my_test_data/test_data` where `my_test_data` is a folder containing the two g-zipped files that each start with `test_data` +`./e-mission-py bin/debug/load_multi_timeline_for_range.py [path/to/files/with/prefix]` -- this path might look something like `/Users/name/Downloads/my_test_data/test_data` where `my_test_data` is a folder containing the two g-zipped files that each start with `test_data` The data will automatically create a "user email" for this opcode, by default it will be "user-0". This can be updated with: `edb.get_uuid_db().update_one({"user_email": "user-0"}, {"$set": {"user_email": "nrelop_usaid-laos-ev_default_test-1"}})` @@ -27,7 +27,7 @@ Then, process the data by running the pipeline. If you updated to a valid opcode, you can now start the server and log into the devapp with that opcode -## Not on Working with Deployment Configs +## Note on Working with Deployment Configs There are development configs designed to be used with the local server. They include `dev-emulator-program`, `dev-emulator-study`, and `dev-emulator-timeuse`. From b221d8910b4eab3250930f36ea1d079cb1f91ee6 Mon Sep 17 00:00:00 2001 From: Abby Wheelis <54848919+Abby-Wheelis@users.noreply.github.com> Date: Thu, 12 Oct 2023 16:14:24 -0600 Subject: [PATCH 3/4] more updates after reading it again --- docs/dev/front/loading_data_for_emulator.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/docs/dev/front/loading_data_for_emulator.md b/docs/dev/front/loading_data_for_emulator.md index 6a1cc61..0ffc907 100644 --- a/docs/dev/front/loading_data_for_emulator.md +++ b/docs/dev/front/loading_data_for_emulator.md @@ -19,7 +19,7 @@ Sometimes, you will have specific data that you want to load into the emulator. `./e-mission-py bin/debug/load_multi_timeline_for_range.py [path/to/files/with/prefix]` -- this path might look something like `/Users/name/Downloads/my_test_data/test_data` where `my_test_data` is a folder containing the two g-zipped files that each start with `test_data` -The data will automatically create a "user email" for this opcode, by default it will be "user-0". This can be updated with: `edb.get_uuid_db().update_one({"user_email": "user-0"}, {"$set": {"user_email": "nrelop_usaid-laos-ev_default_test-1"}})` +The script will automatically create a "user email" for this opcode, by default it will be "user-0". This can be updated with: `edb.get_uuid_db().update_one({"user_email": "user-0"}, {"$set": {"user_email": "nrelop_[valid opcode]"}})` Then, process the data by running the pipeline. From e896dd16e263b362fd4797b1e836b93d89dc8a13 Mon Sep 17 00:00:00 2001 From: Abby Wheelis <54848919+Abby-Wheelis@users.noreply.github.com> Date: Sat, 14 Oct 2023 17:36:47 -0400 Subject: [PATCH 4/4] resolve confusion around deployment vs development --- docs/dev/front/loading_data_for_emulator.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/docs/dev/front/loading_data_for_emulator.md b/docs/dev/front/loading_data_for_emulator.md index 0ffc907..7566d83 100644 --- a/docs/dev/front/loading_data_for_emulator.md +++ b/docs/dev/front/loading_data_for_emulator.md @@ -35,7 +35,7 @@ Sometimes, it is useful to load data into a deployment config for testing, demo, 1. Fork `nrel-openpath-deploy-configs` 2. Modify the config you want to work on, making sure that there is no `server` defined (if no server is defined, the app will connect to the local server, as desired). 3. Update the downloadURL in `dynamicConfig` for the version of `e-mission-phone` you are running to `https://raw.githubusercontent.com/[MY-REPO]/nrel-openpath-deploy-configs/[MY-BRANCH]/configs/${label}.nrel-op.json` -4. Now you can use a development config with the local server +4. Now you can use a deployment config with the local server ## Notes on making up a valid opcode - opcodes must begin with `nrelop`