Skip to content

Commit

Permalink
deploy: 97178f1
Browse files Browse the repository at this point in the history
  • Loading branch information
e-marshall committed Feb 4, 2024
1 parent 583c13e commit 7f97e63
Show file tree
Hide file tree
Showing 3 changed files with 55 additions and 46 deletions.
23 changes: 17 additions & 6 deletions _sources/asf_local_mf.ipynb
Original file line number Diff line number Diff line change
Expand Up @@ -46,16 +46,15 @@
"import pandas as pd\n",
"import dask\n",
"\n",
"from pathlib import Path\n",
"from intake.source.utils import reverse_format\n"
"from pathlib import Path"
]
},
{
"cell_type": "markdown",
"id": "deaa7cdb-2e90-4fc3-a944-84dcfdf6953e",
"metadata": {},
"source": [
"We'll use this function later but defining it now:"
"We'll use this function later but define it now:"
]
},
{
Expand Down Expand Up @@ -278,7 +277,7 @@
"source": [
"from dask.distributed import Client\n",
"\n",
"client = Client(processes=False) #look into limiting memory\n",
"client = Client(processes=False) \n",
"client"
]
},
Expand Down Expand Up @@ -507,7 +506,7 @@
"id": "71c3cb5b-c6bd-46ad-b6e2-c3c531a6cebb",
"metadata": {},
"source": [
"Repeat for the other two variables (unless these can be combined into one preprocess?)"
"Repeat for the other two variables "
]
},
{
Expand Down Expand Up @@ -677,7 +676,7 @@
"id": "3c022c13-c49f-4b56-9463-49843294c934",
"metadata": {},
"source": [
"First, let's call `xr.open_mfdataset()` with the argument `chunks='auto'`. This will read in a dask array where ideal chunk sizes are selected based off the array size, it will attempt to have chunk sizes where bytes are equal to the configuration value for array chunk size. More about that [here](https://docs.dask.org/en/stable/array-chunks.html#automatic-chunking).You can check the configuration value for an array chunk size with the code below: "
"First, let's call `xr.open_mfdataset()` with the argument `chunks='auto'`. This will read in a dask array where ideal chunk sizes are selected based off the array size, it will attempt to have chunk sizes where bytes are equal to the configuration value for array chunk size. More about that [here](https://docs.dask.org/en/stable/array-chunks.html#automatic-chunking). You can check the configuration value for an array chunk size with the code below: "
]
},
{
Expand Down Expand Up @@ -5645,6 +5644,18 @@
"display_name": "Python 3 (ipykernel)",
"language": "python",
"name": "python3"
},
"language_info": {
"codemirror_mode": {
"name": "ipython",
"version": 3
},
"file_extension": ".py",
"mimetype": "text/x-python",
"name": "python",
"nbconvert_exporter": "python",
"pygments_lexer": "ipython3",
"version": "3.10.13"
}
},
"nbformat": 4,
Expand Down
Loading

0 comments on commit 7f97e63

Please sign in to comment.