Update from main to avoid conflicts (#212)

* Change setup to fix MIRA name error (#205) * changed mira version before bug * fix error * Add utilities for loading distributions from AMR (#200) * added mira distribution loading * added normal distribution * fixed Normal2 and Normal3 * nit * added minimal mira_distribution_to_pyro test * Symbolic Rate law to Pytorch Rate law (#201) * I believe I wrote the correct code, based on experiments in the notebook. Will test next. * FAILED test/test_mira/test_rate_law.py::TestRateLaw::test_rate_law_compilation - AttributeError: 'ScaledBetaNoisePetriNetODESystem' object has no attribute 'beta' * Added Symbolic_Deriv_Experiments notebook * Something weird is happening. I can confirm that 'beta' is an attribute of ScaledBetaNoisePetriNetODESystem after setting up the model, but then it can't be found at sample time * Clarified the bug in the Symbolic derivatives notebook * Expected and actual derivative match * Time varying parameter rate law correctly read * Thought we added this already * Added kwargs to from_askenet and from_mira and compile_rate_law_p to load_petri_net * Blocked on gyorilab/mira#189 but tests pass by making compile_rate_law_p False by default * Removed unnecessary pygraphviz dependency * Unit test to fail when concept name does not equal rate law symbols * All tests pass with default compile_rate_law_p = False * Merged from main. removed dependency on older version of mira * point mira to the github repo main branch * point mira to the github repo main branch * load_and_calibrate_and_sample(..., compile_rate_law_p=True) works with the caveat that the ScaledBetaNoisePetriNetODESystem solution was returning very slightly negative values, so I set mean = torch.abs(solution[var_name]) to address the issue * merged changes to MiraPetriNetODESystem and ScaledBetaNoisePetriNetODESystem from main. ScaledBetaNoisePetriNetODESystem has default compiled_rate_law_p=True * observation_function for ScaledBetaNoisePetriNetODESystem now uses torch.maximum(solution[var_name], torch.tensor(1e-9)) to deal with overshooting derivatives * aggregate parameters is now by default opt-out, and AMR models with multiple parameters per transition can be interpreted using compile_rate_law * 12-Month Hackathon Notebooks (#207) * started hackathon prep scenario notebooks * more on hackathon scenarios * more work on hackathon notebooks * created AMR for scenario1 with constant beta * Vs hackathon prep (#203) * beginning of ensemble challenge ipynb * Update ensemble_challenge.ipynb * added to scenario2 notebook, ready for scenario3 * added to scenario2 notebook, ready for scenario3 * Update scenario1.ipynb * Update scenario1 * changes to scenario3, and new AMR * pre-hackathon prep update - scenario1 task 1 almost done (pending AMR changes) - ensemble challenge layout started * more on scenario notebooks, added AMR to scenario2 * Fixed the nan inside the intervened parameters column output * updates to scenario3 notebook * merged from main --------- Co-authored-by: vsella <vsella@utexas.edu> Co-authored-by: Jeremy Zucker <Jeremy.Zucker@pnnl.gov> --------- Co-authored-by: Sam Witty <samawitty@gmail.com> Co-authored-by: Jeremy Zucker <djinnome@gmail.com> Co-authored-by: sabinala <130604122+sabinala@users.noreply.github.com> Co-authored-by: vsella <vsella@utexas.edu> Co-authored-by: Jeremy Zucker <Jeremy.Zucker@pnnl.gov>
ciemss · Jul 10, 2023 · 6b77a9d · 6b77a9d
1 parent 238b49d
commit 6b77a9d
Show file tree

Hide file tree

Showing 12 changed files with 2,663 additions and 6 deletions.
diff --git a/notebook/hackathon_prep/ensemble_challenge.ipynb b/notebook/hackathon_prep/ensemble_challenge.ipynb
@@ -0,0 +1,129 @@
+{
+ "cells": [
+ {
+ "cell_type": "markdown",
+ "metadata": {},
+ "source": [
+ "# Ensemble Challenge"
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "metadata": {},
+ "source": [
+ "For each timepoint, consider the following questions and complete the associated tasks:\n",
+ "1. Timepoint 1: May 1st, 2020, location = Michigan state, at the beginning of the pandemic,\n",
+ "when the main preventive measure was masking. No vaccines were available during this\n",
+ "time\n",
+ "2. Timepoint 2: May 1st, 2021, location = Michigan state, prior to the Covid wave caused by\n",
+ "the arrival of the Delta variant. Vaccines were available at this time\n",
+ "3. Timepoint 3: December 15th, 2021, location = Michigan state, during the start of the\n",
+ "winter Covid wave caused by the original Omicron variant\n",
+ "\n",
+ "(Bonus: For all 3 timepoints, now consider Louisiana, which had different Covid-19 dynamics\n",
+ "compared to the northern and northeastern states)"
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "metadata": {},
+ "source": [
+ "## Exploratory Work"
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "metadata": {},
+ "source": [
+ "For each timepoint, consider the following during model and data selection:\n",
+ "\n",
+ "TA? Task 1: What is the most relevant data to use for model calibration?\n",
+ "\n",
+ "TA? Task 2: What was our understanding of Covid-19 viral mechanisms at the time?\n",
+ "\n",
+ "TA? Task 3: What are the values of parameter values related to contagiousness/transmissibility, and\n",
+ "severity of the dominant strain at the time?\n",
+ "\n",
+ "TA? Task 4: What policies were in place for a stated location, and how can this information be\n",
+ "incorporated into models?"
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "metadata": {},
+ "source": [
+ "## Part 1"
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "metadata": {},
+ "source": [
+ "For each timepoint:"
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "metadata": {},
+ "source": [
+ "**TA3 Task 1** : Take a single model, calibrate it using any historical data prior to the\n",
+ "given date, and predict cases, hospitalizations, and deaths over a 4-week timeframe\n",
+ "beginning from the given date, as done in the CDC forecasting challenge. Evaluate the\n",
+ "forecast using the Covid-19 forecasting hub error metrics (WIS, MAE) described below.\n",
+ "(Note: The single model forecast should be evaluated in the same way as the ensemble)\n",
+ "\n",
+ "Information on WIS: https://arxiv.org/abs/2005.12881\n",
+ "\n",
+ "Information on ensemble evaluation method: https://www.medrxiv.org/content/10.1101/2021.02.03.21250974v1\n"
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "metadata": {},
+ "source": [
+ "## Part 2"
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "metadata": {},
+ "source": [
+ "**(A)/(B) TA3 Task 2**: For the ensemble, it is fine to calibrate each model independently and weight\n",
+ "them naively based on their skill. Or, it would also be fine to calibrate the ensemble as a whole, assigning weights to\n",
+ "the different component models, so that you minimize the error of the ensemble on the historical data. \n",
+ "\n",
+ "Use the calibration scores and error metrics computed by the CDC forecasting\n",
+ "hub. "
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "metadata": {},
+ "source": [
+ "**(C) TA3 Task 3**: Produce the forecast outputs in the format specified by the CDC forecasting challenge,\n",
+ "including the specified quantiles."
+ ]
+ }
+ ],
+ "metadata": {
+ "kernelspec": {
+ "display_name": "Python 3 (ipykernel)",
+ "language": "python",
+ "name": "python3"
+ },
+ "language_info": {
+ "codemirror_mode": {
+ "name": "ipython",
+ "version": 3
+ },
+ "file_extension": ".py",
+ "mimetype": "text/x-python",
+ "name": "python",
+ "nbconvert_exporter": "python",
+ "pygments_lexer": "ipython3",
+ "version": "3.9.6"
+ }
+ },
+ "nbformat": 4,
+ "nbformat_minor": 2
+}
diff --git a/notebook/hackathon_prep/hackathon_models_to_amr.ipynb b/notebook/hackathon_prep/hackathon_models_to_amr.ipynb
@@ -0,0 +1,181 @@
+{
+ "cells": [
+ {
+ "cell_type": "markdown",
+ "id": "2d26a345",
+ "metadata": {},
+ "source": [
+ "# Produce ASKEM Model Representations of models to be used during the 12-month Hackathon"
+ ]
+ },
+ {
+ "cell_type": "code",
+ "execution_count": 5,
+ "id": "6d523e3d",
+ "metadata": {},
+ "outputs": [],
+ "source": [
+ "import sympy\n",
+ "from mira.metamodel import *\n",
+ "from mira.modeling import Model\n",
+ "from mira.modeling.askenet.petrinet import AskeNetPetriNetModel"
+ ]
+ },
+ {
+ "cell_type": "code",
+ "execution_count": 10,
+ "id": "a94a4534",
+ "metadata": {},
+ "outputs": [],
+ "source": [
+ "person_units = Unit(expression=sympy.Symbol('person'))\n",
+ "day_units = Unit(expression=sympy.Symbol('day'))\n",
+ "per_day_units = Unit(expression=1/sympy.Symbol('day'))\n",
+ "dimensionless_units = Unit(expression=sympy.Integer('1'))\n",
+ "\n",
+ "c = {\n",
+ " 'S': Concept(name='S', units=person_units),\n",
+ " 'E': Concept(name='E', units=person_units),\n",
+ " 'I': Concept(name='I', units=person_units),\n",
+ " 'R': Concept(name='R', units=person_units),\n",
+ " 'D': Concept(name='D', units=person_units)\n",
+ "}\n",
+ "\n",
+ "\n",
+ "parameters = {\n",
+ " 'gamma': Parameter(name='gamma', value=1/11, units=per_day_units),\n",
+ " 'delta': Parameter(name='delta', value=1/5, units=per_day_units),\n",
+ " 'alpha': Parameter(name='alpha', value=0.000064, units=dimensionless_units),\n",
+ " 'rho': Parameter(name='rho', value=1/9, units=per_day_units),\n",
+ " 'N': Parameter(name='N', value=5_600_000, units=person_units),\n",
+ " 'beta_s': Parameter(name='beta_s', value=1),\n",
+ " 'beta_c': Parameter(name='beta_c', value=0.4),\n",
+ " 't_0': Parameter(name='t_0', value=89, unts=day_units),\n",
+ " # D=11, gamma = 1/D, R_0 = 5 and\n",
+ " # beta = R_0 * gamma * mask(t) so kappa = 5/11\n",
+ " 'kappa': Parameter(name='kappa', value=5/11),\n",
+ "}\n",
+ "\n",
+ "initials = {\n",
+ " 'S': Initial(concept=Concept(name='S'), value=5_600_000-1),\n",
+ " 'E': Initial(concept=Concept(name='E'), value=1),\n",
+ " 'I': Initial(concept=Concept(name='I'), value=0),\n",
+ " 'R': Initial(concept=Concept(name='R'), value=0),\n",
+ " 'D': Initial(concept=Concept(name='D'), value=0),\n",
+ "}"
+ ]
+ },
+ {
+ "cell_type": "code",
+ "execution_count": 7,
+ "id": "3731a873",
+ "metadata": {},
+ "outputs": [],
+ "source": [
+ "S, E, I, R, D, N, kappa, beta_s, beta_c, k, t_0, t, alpha, delta, rho, gamma = \\\n",
+ " sympy.symbols('S E I R D N kappa beta_s beta_c k t_0 t alpha delta rho gamma')"
+ ]
+ },
+ {
+ "cell_type": "code",
+ "execution_count": 8,
+ "id": "2a4a8123",
+ "metadata": {},
+ "outputs": [],
+ "source": [
+ "beta = kappa*beta_s\n",
+ "\n",
+ "t1 = ControlledConversion(subject=c['S'],\n",
+ " outcome=c['E'],\n",
+ " controller=c['E'],\n",
+ " rate_law=S*I*beta / N)\n",
+ "t2 = NaturalConversion(subject=c['E'],\n",
+ " outcome=c['I'],\n",
+ " rate_law=delta*E)\n",
+ "t3 = NaturalConversion(subject=c['I'],\n",
+ " outcome=c['R'],\n",
+ " rate_law=(1-alpha)*gamma*I)\n",
+ "t4 = NaturalConversion(subject=c['I'],\n",
+ " outcome=c['D'],\n",
+ " rate_law=alpha*rho*I)\n",
+ "templates = [t1, t2, t3, t4]\n",
+ "tm = TemplateModel(\n",
+ " templates=templates,\n",
+ " parameters=parameters,\n",
+ " initials=initials,\n",
+ " time=Time(name='t', units=day_units)\n",
+ ")\n",
+ "AskeNetPetriNetModel(Model(tm)).to_json_file('scenario1_constant_beta.json')"
+ ]
+ },
+ {
+ "cell_type": "code",
+ "execution_count": 9,
+ "id": "72681fce",
+ "metadata": {},
+ "outputs": [],
+ "source": [
+ "beta = kappa*beta_s\n",
+ "\n",
+ "t1 = ControlledConversion(subject=c['S'],\n",
+ " outcome=c['E'],\n",
+ " controller=c['I'],\n",
+ " rate_law=S*I*beta / N)\n",
+ "t2 = NaturalConversion(subject=c['E'],\n",
+ " outcome=c['I'],\n",
+ " rate_law=delta*E)\n",
+ "t3 = NaturalConversion(subject=c['I'],\n",
+ " outcome=c['R'],\n",
+ " rate_law=(1-alpha)*gamma*I)\n",
+ "t4 = NaturalConversion(subject=c['I'],\n",
+ " outcome=c['D'],\n",
+ " rate_law=alpha*rho*I)\n",
+ "templates = [t1, t2, t3, t4]\n",
+ "tm = TemplateModel(\n",
+ " templates=templates,\n",
+ " parameters=parameters,\n",
+ " initials=initials,\n",
+ " time=Time(name='t', units=day_units)\n",
+ ")\n",
+ "AskeNetPetriNetModel(Model(tm)).to_json_file('scenario1_constant_beta_try_again.json')"
+ ]
+ },
+ {
+ "cell_type": "code",
+ "execution_count": null,
+ "id": "32a788d6",
+ "metadata": {},
+ "outputs": [],
+ "source": []
+ },
+ {
+ "cell_type": "code",
+ "execution_count": null,
+ "id": "4c262416",
+ "metadata": {},
+ "outputs": [],
+ "source": []
+ }
+ ],
+ "metadata": {
+ "kernelspec": {
+ "display_name": "Python 3 (ipykernel)",
+ "language": "python",
+ "name": "python3"
+ },
+ "language_info": {
+ "codemirror_mode": {
+ "name": "ipython",
+ "version": 3
+ },
+ "file_extension": ".py",
+ "mimetype": "text/x-python",
+ "name": "python",
+ "nbconvert_exporter": "python",
+ "pygments_lexer": "ipython3",
+ "version": "3.10.9"
+ }
+ },
+ "nbformat": 4,
+ "nbformat_minor": 5
+}
diff --git a/notebook/hackathon_prep/scenario1.ipynb b/notebook/hackathon_prep/scenario1.ipynb
diff --git a/notebook/hackathon_prep/scenario1_constant_beta.json b/notebook/hackathon_prep/scenario1_constant_beta.json
@@ -0,0 +1 @@
+{"name": "Model", "schema": "https://raw.githubusercontent.com/DARPA-ASKEM/Model-Representations/petrinet_v0.5/petrinet/petrinet_schema.json", "schema_name": "petrinet", "description": "Model", "model_version": "0.1", "properties": {}, "model": {"states": [{"id": "S", "name": "S", "grounding": {"identifiers": {}, "modifiers": {}}, "units": {"expression": "person", "expression_mathml": "<ci>person</ci>"}}, {"id": "E", "name": "E", "grounding": {"identifiers": {}, "modifiers": {}}, "units": {"expression": "person", "expression_mathml": "<ci>person</ci>"}}, {"id": "I", "name": "I", "grounding": {"identifiers": {}, "modifiers": {}}, "units": {"expression": "person", "expression_mathml": "<ci>person</ci>"}}, {"id": "R", "name": "R", "grounding": {"identifiers": {}, "modifiers": {}}, "units": {"expression": "person", "expression_mathml": "<ci>person</ci>"}}, {"id": "D", "name": "D", "grounding": {"identifiers": {}, "modifiers": {}}, "units": {"expression": "person", "expression_mathml": "<ci>person</ci>"}}], "transitions": [{"id": "t1", "input": ["E", "S"], "output": ["E", "E"], "properties": {"name": "t1"}}, {"id": "t2", "input": ["E"], "output": ["I"], "properties": {"name": "t2"}}, {"id": "t3", "input": ["I"], "output": ["R"], "properties": {"name": "t3"}}, {"id": "t4", "input": ["I"], "output": ["D"], "properties": {"name": "t4"}}]}, "semantics": {"ode": {"rates": [{"target": "t1", "expression": "I*S*beta_s*kappa/N", "expression_mathml": "<apply><divide/><apply><times/><ci>I</ci><ci>S</ci><ci>beta_s</ci><ci>kappa</ci></apply><ci>N</ci></apply>"}, {"target": "t2", "expression": "E*delta", "expression_mathml": "<apply><times/><ci>E</ci><ci>delta</ci></apply>"}, {"target": "t3", "expression": "I*gamma*(1 - alpha)", "expression_mathml": "<apply><times/><ci>I</ci><ci>gamma</ci><apply><minus/><cn>1</cn><ci>alpha</ci></apply></apply>"}, {"target": "t4", "expression": "I*alpha*rho", "expression_mathml": "<apply><times/><ci>I</ci><ci>alpha</ci><ci>rho</ci></apply>"}], "initials": [{"target": "S", "expression": "5599999.00000000", "expression_mathml": "<cn>5599999.0</cn>"}, {"target": "E", "expression": "1.00000000000000", "expression_mathml": "<cn>1.0</cn>"}, {"target": "I", "expression": "0.0", "expression_mathml": "<cn>0.0</cn>"}, {"target": "R", "expression": "0.0", "expression_mathml": "<cn>0.0</cn>"}, {"target": "D", "expression": "0.0", "expression_mathml": "<cn>0.0</cn>"}], "parameters": [{"id": "N", "value": 5600000.0, "units": {"expression": "person", "expression_mathml": "<ci>person</ci>"}}, {"id": "beta_s", "value": 1.0}, {"id": "kappa", "value": 0.45454545454545453}, {"id": "delta", "value": 0.2, "units": {"expression": "1/day", "expression_mathml": "<apply><power/><ci>day</ci><cn>-1</cn></apply>"}}, {"id": "gamma", "value": 0.09090909090909091, "units": {"expression": "1/day", "expression_mathml": "<apply><power/><ci>day</ci><cn>-1</cn></apply>"}}, {"id": "alpha", "value": 6.4e-05, "units": {"expression": "1", "expression_mathml": "<cn>1</cn>"}}, {"id": "rho", "value": 0.1111111111111111, "units": {"expression": "1/day", "expression_mathml": "<apply><power/><ci>day</ci><cn>-1</cn></apply>"}}], "observables": [], "time": {"id": "t", "units": {"expression": "day", "expression_mathml": "<ci>day</ci>"}}}}, "metadata": {"annotations": {"license": null, "authors": [], "references": [], "time_scale": null, "time_start": null, "time_end": null, "locations": [], "pathogens": [], "diseases": [], "hosts": [], "model_types": []}}}
diff --git a/notebook/hackathon_prep/scenario1_constant_beta_try_again.json b/notebook/hackathon_prep/scenario1_constant_beta_try_again.json
@@ -0,0 +1 @@
+{"name": "Model", "schema": "https://raw.githubusercontent.com/DARPA-ASKEM/Model-Representations/petrinet_v0.5/petrinet/petrinet_schema.json", "schema_name": "petrinet", "description": "Model", "model_version": "0.1", "properties": {}, "model": {"states": [{"id": "S", "name": "S", "grounding": {"identifiers": {}, "modifiers": {}}, "units": {"expression": "person", "expression_mathml": "<ci>person</ci>"}}, {"id": "I", "name": "I", "grounding": {"identifiers": {}, "modifiers": {}}, "units": {"expression": "person", "expression_mathml": "<ci>person</ci>"}}, {"id": "E", "name": "E", "grounding": {"identifiers": {}, "modifiers": {}}, "units": {"expression": "person", "expression_mathml": "<ci>person</ci>"}}, {"id": "R", "name": "R", "grounding": {"identifiers": {}, "modifiers": {}}, "units": {"expression": "person", "expression_mathml": "<ci>person</ci>"}}, {"id": "D", "name": "D", "grounding": {"identifiers": {}, "modifiers": {}}, "units": {"expression": "person", "expression_mathml": "<ci>person</ci>"}}], "transitions": [{"id": "t1", "input": ["I", "S"], "output": ["I", "E"], "properties": {"name": "t1"}}, {"id": "t2", "input": ["E"], "output": ["I"], "properties": {"name": "t2"}}, {"id": "t3", "input": ["I"], "output": ["R"], "properties": {"name": "t3"}}, {"id": "t4", "input": ["I"], "output": ["D"], "properties": {"name": "t4"}}]}, "semantics": {"ode": {"rates": [{"target": "t1", "expression": "I*S*beta_s*kappa/N", "expression_mathml": "<apply><divide/><apply><times/><ci>I</ci><ci>S</ci><ci>beta_s</ci><ci>kappa</ci></apply><ci>N</ci></apply>"}, {"target": "t2", "expression": "E*delta", "expression_mathml": "<apply><times/><ci>E</ci><ci>delta</ci></apply>"}, {"target": "t3", "expression": "I*gamma*(1 - alpha)", "expression_mathml": "<apply><times/><ci>I</ci><ci>gamma</ci><apply><minus/><cn>1</cn><ci>alpha</ci></apply></apply>"}, {"target": "t4", "expression": "I*alpha*rho", "expression_mathml": "<apply><times/><ci>I</ci><ci>alpha</ci><ci>rho</ci></apply>"}], "initials": [{"target": "S", "expression": "5599999.00000000", "expression_mathml": "<cn>5599999.0</cn>"}, {"target": "I", "expression": "0.0", "expression_mathml": "<cn>0.0</cn>"}, {"target": "E", "expression": "1.00000000000000", "expression_mathml": "<cn>1.0</cn>"}, {"target": "R", "expression": "0.0", "expression_mathml": "<cn>0.0</cn>"}, {"target": "D", "expression": "0.0", "expression_mathml": "<cn>0.0</cn>"}], "parameters": [{"id": "N", "value": 5600000.0, "units": {"expression": "person", "expression_mathml": "<ci>person</ci>"}}, {"id": "beta_s", "value": 1.0}, {"id": "kappa", "value": 0.45454545454545453}, {"id": "delta", "value": 0.2, "units": {"expression": "1/day", "expression_mathml": "<apply><power/><ci>day</ci><cn>-1</cn></apply>"}}, {"id": "gamma", "value": 0.09090909090909091, "units": {"expression": "1/day", "expression_mathml": "<apply><power/><ci>day</ci><cn>-1</cn></apply>"}}, {"id": "alpha", "value": 6.4e-05, "units": {"expression": "1", "expression_mathml": "<cn>1</cn>"}}, {"id": "rho", "value": 0.1111111111111111, "units": {"expression": "1/day", "expression_mathml": "<apply><power/><ci>day</ci><cn>-1</cn></apply>"}}], "observables": [], "time": {"id": "t", "units": {"expression": "day", "expression_mathml": "<ci>day</ci>"}}}}, "metadata": {"annotations": {"license": null, "authors": [], "references": [], "time_scale": null, "time_start": null, "time_end": null, "locations": [], "pathogens": [], "diseases": [], "hosts": [], "model_types": []}}}
diff --git a/notebook/hackathon_prep/scenario2.ipynb b/notebook/hackathon_prep/scenario2.ipynb
diff --git a/notebook/hackathon_prep/scenario3_abm.ipynb b/notebook/hackathon_prep/scenario3_abm.ipynb