Merge pull request #112 from gaelforget/v0p3p0b

V0p3p0b

Project.toml

@@ -1,7 +1,7 @@
 name = "ClimateModels"
 uuid = "f6adb021-9183-4f40-84dc-8cea6f651bb0"
 authors = ["gaelforget <gforget@mit.edu>"]
-version = "0.2.21"
+version = "0.3.0"
 
 [deps]
 CSV = "336ed68f-0bac-5ca0-87d4-7b16caf5d00b"

docs/src/API.md

@@ -3,16 +3,12 @@
 
 ## Data Structures
 
-`ModelConfig` is the main concrete type of `AbstractModelConfig`.
+- `ModelConfig` is the main concrete type of `AbstractModelConfig`; used in the [examples](@ref examples).
+- `PlutoConfig` let's us ingest any [Pluto.jl](https://github.com/fonsp/Pluto.jl/wiki) notebook easily via `ClimateModels`' [Notebooks Methods](@ref notebook_methods).
 
 ```@docs
 ModelConfig
 ModelConfig(::Function)
-```
-
-`PlutoConfig` is also concrete type of `AbstractModelConfig`. It is provided to use [Pluto.jl](https://github.com/fonsp/Pluto.jl/wiki) notebooks via the interface (see [Notebooks Methods](@ref notebook_methods)).
-
-```@docs
 PlutoConfig
 ```
 
@@ -25,34 +21,34 @@ launch
 log
 ```
 
-Also provided : [`pathof`](@ref), [`joinpath`](@ref), [`readdir`](@ref), [`show`](@ref), [`clean`](@ref), and [`@ModelRun`](@ref)
+Also provided : [`pathof`](@ref), [`joinpath`](@ref), [`cd`](@ref), [`readdir`](@ref), [`show`](@ref), [`clean`](@ref), and [`@ModelRun`](@ref)
 
 ## [Notebook Methods](@id notebook_methods)
 
-The `setup` method for `PlutoConfig` uses `unroll` to pre-process notebook codes.
-
 ```@docs
 setup(::PlutoConfig)
-notebooks.update(::PlutoConfig)
+update(::PlutoConfig)
 notebooks.open(::PlutoConfig)
-notebooks.unroll
-notebooks.reroll
 ```
 
+!!! note
+ `setup` and `update` use `unroll` and `reroll` internally to process notebooks.
+
 ## More
 
 ### Simplified API
 
 ```@docs
-@ModelRun
 ModelRun
+@ModelRun
 ```
 
 ### Utility Functions
 
 ```@docs
 pathof
 joinpath
+cd
 readdir
 show
 clean

docs/src/examples.md

@@ -1,21 +1,19 @@
 # [Examples](@id examples)
 
-A good place to start is the [random walk model](RandomWalker.html) example. It is also presented in greater detail in the [Climate Model Interface](@ref) section to further illustrate how things work.
-
-```@docs
-RandomWalker
-```
-
-The examples broadly fall into two categories : 
+A good place to start is the [random walk model](RandomWalker.html) example, which is used in the [Climate Model Interface](@ref) documentation. The other examples, below, fall more or less into two categories : 
 
 - [Workflows That Run Models](@ref run_model_examples)
 - [Workflows That Replay Models](@ref replay_model_examples)
 
-For each model listed below, the main language or file format is indicated. Models are sorted by increasing problem size. Thus, the included notebook collection illustrates that the [Climate Model Interface](@ref run_model_examples) is applicable to a wide range of model types, programming languages, and problem sizes.
+The main language and file format of the model examples vary. The notebook collection shows how [Climate Model Interface](@ref run_model_examples) is easily to a wide range of model types, programming languages, and problem sizes.
 
 In [User Manual](@ref manual), the [Climate Model Interface](@ref) section outlines several simple ways that models can be added to the framework. The examples presented here were built in this fashion.
 
-[Trying Out The Examples](@ref) is geared toward users who may want to run or experiment with models. 
+[Trying Out The Examples](@ref) is geared toward users who may want to experiment with models. 
+
+```@docs
+RandomWalker
+```
 
 ## [Workflows That Run Models](@id run_model_examples)
 
@@ -35,6 +33,15 @@ In [User Manual](@ref manual), the [Climate Model Interface](@ref) section outli
 - [ECCO version 4](https://JuliaOcean.github.io/OceanStateEstimation.jl/dev/examples/ECCO_standard_plots.html) (NetCDF) ➭ [code link](https://raw.githubusercontent.com/gaelforget/OceanStateEstimation.jl/master/examples/ECCO/ECCO_standard_plots.jl)
 - [Pathway Simulations](https://gaelforget.github.io/MITgcmTools.jl/dev/examples/HS94_particles.html) (binary, jld2) ➭ [code link](https://raw.githubusercontent.com/gaelforget/MITgcmTools.jl/master/examples/HS94_particles.jl)
 
+
+## JuliaCon 2021 Presentation
+
+- [notebook view (html)](ClimateModelsJuliaCon2021.html)
+- [notebook source (jl)](https://github.com/gaelforget/ClimateModels.jl/blob/master/docs/src/ClimateModelsJuliaCon2021.jl)
+- [video recording (mp4)](https://youtu.be/XR5hKCja0uw)
+
+[![Screen Shot 2021-08-31 at 2 25 04 PM](https://user-images.githubusercontent.com/20276764/131556274-48f3df13-0608-4cd0-acf9-c3e29894a32c.png)](https://youtu.be/XR5hKCja0uw)
+
 ## Trying Out The Examples
 
 The examples can be most easy to run using [Pluto.jl](https://github.com/fonsp/Pluto.jl). The [JuliaClimate/Notebooks](https://juliaclimate.github.io/Notebooks) webpage links to free cloud resources and directions to run notebooks on your own computer.

docs/src/functionalities.md

@@ -17,53 +17,51 @@ tmp=joinpath(tempdir(),"notebook.jl")
 cp(fil,tmp,force=true)
 ```
 
-Here we document key functionalities offered in `ClimateModels.jl`
-
-- Climate model interface
-- Pluto notebook integration
-- Tracked worklow framework
-- File access and cloud support
-- Plotting recipes and examples
+Here are key functionalities offered in `ClimateModels.jl`.
 
 ## Climate Model Interface
 
 The interface ties the [`ModelConfig`](@ref) data structure with methods like [`setup`](@ref), [`build`](@ref), and [`launch`](@ref). In return, it provides standard methods to deal with inputs and outputs, as well as capabilities described below. 
 
-The [`ModelRun`](@ref) method provides the capability to deploy models in streamlined fashion -- with just one code line, or just one click. It executes all three steps at once ([`setup`](@ref), [`build`](@ref), and [`launch`](@ref)). 
-
-For example, let's use [`RandomWalker`](@ref) as the model main function. 
+The [`ModelRun`](@ref) method, or just `run`, streamlines the process. It executes all three steps at once ([`setup`](@ref), [`build`](@ref), and [`launch`](@ref)). For example, let's use [`RandomWalker`](@ref) as the model.
 
 ```@example 1
 fun=ClimateModels.RandomWalker
+nothing # hide
 ```
 
-With the simplified [`ModelConfig`](@ref) constructor, we can then just write any of the following:
+With the simplified [`ModelConfig`](@ref) constructors, we can just write any of the following:
 
-```@example 2
+```
 ModelRun(ModelConfig(model=fun))
-run(ModelConfig(fun))
-nothing # hide
 ```
 
-Or via the `@ModelRun` macro:
+or 
+
+```@example 2
+MC=run(ModelConfig(fun))
+log(MC)
+```
+
+or
 
 ```@example 1
 @ModelRun ClimateModels.RandomWalker
-nothing # hide
 ```
 
-By design of our interface, **it is required** that function `fun` receives a `ModelConfig` as its sole input argument. 
+By design of the `ClimateModels` interface, **it is required** that `fun` receives a `ModelConfig` as its sole input argument. **This requirement is easily satisfied** in practice. 
 
-In practice, **this requirement is easily satisfied**. Input parameters can be specified to `ModelConfig` via the `inputs` keyword argument, or via files instead. See [Parameters](@ref).
+Input parameters can be specified via the `inputs` keyword argument, or via files. See [Parameters](@ref).
 
-Often one may prefer to break things down though. Let's start with defining the model:
+### Breaking Things Down
+
+Let's start with defining the model:
 
 ```@example 2
 MC=ModelConfig(model=fun)
-nothing # hide
 ```
 
-The sequence of calls within `ModelRun` can then be expanded as shown below. In practice, `setup` typically handles files and software, `build` may compile a chosen model configuration, and `launch` takes care of the main computation. 
+The sequence of calls within `ModelRun` is expanded below. In practice, `setup` typically handles files and software, `build` gets the model ready, and `launch` starts the model computation. 
 
 ```@example 2
 setup(MC)
@@ -88,7 +86,7 @@ pathof(MC)
 readdir(MC)
 ```
 
-The `log` subfolder was created earlier by [`setup`](@ref). The [`log`](@ref) function retrieves the workflow log. 
+The `log` subfolder was created earlier by [`setup`](@ref). The [`log`](@ref) function can then retrieve the workflow log. 
 
 ```@example 2
 log(MC)
@@ -127,7 +125,7 @@ As shown in the [Parameters](@ref) example:
 
 ### Parameters
 
-In this example, we illustrate how one can interact with model parameters, rerun a model, and keep track of these workflow steps,
+Let's now mofdify model parameters, then rerun a model, and keep track of these workflow steps.
 
 After an initial model run of 100 steps, duration `NS` is extended to 200 time steps. The [`put!`](@ref) and [`launch`](@ref) sequence then reruns the model. 
 
@@ -165,16 +163,16 @@ Any Pluto notebook is easily integrated to the `ClimateModels.jl` framework via
 
 ```@example 3
 filename=joinpath(tempdir(),"notebook.jl")
-PC=PlutoConfig(filename)
+PC=PlutoConfig(filename,(linked_model="MC",))
 run(PC)
 readdir(PC)
 ```
 
-This functionality reformats the Pluto notebook via [`unroll`](@ref) and runs the notebook code in the notebook environment. All files get copied into `pathof(PC)` as before. This approach provides a simple way to run in batch mode model configurations documented in notebooks. 
+The Pluto notebook gets split up into main code (1) and environment (2). This approach provides a simple way to go from model documentation, in notebook format, to large simulations run, done in batch mode.
 
-If a notebook itself contains a `ModelConfig` called `MC` then the corresponding folder is linked into the `PlutoConfig` folder at the end. This feature (ilustrated just above) makes it easy to trace back where the `MC` output files are.
+Files get copied into `pathof(PC)` as before. If `notebook.jl` contains a `ModelConfig`, let's call it `MC`, then the `pathof(MC)` folder can be linked into `pathof(PC)` at the end. This feature is controlled by `linked_model` as illustrated just before. A data input folder can be specified via the `data_folder` key. This will result in the specified folder getting linked into `pathof(PC)` before running the notebook.
 
-The [`update`](@ref) method for a [`PlutoConfig`](@ref) adds a simple method for updating notebook dependencies. This is a routine maintanance operation, which is often followed by rerunning the notebook to detect potential updating issues.
+[`update`](@ref) provides a simple method for updating notebook dependencies. Such routine maintanance is often followed by rerunning the notebook to detect potential updating issues.
 
 ```@example 3
 update(PlutoConfig(filename))
@@ -184,9 +182,9 @@ nothing # hide
 
 ## Files and Cloud Support
 
-There are various ways that numerical model output gets archived, distributed, and retrieved. In some cases downloading data from the web can be most convenient. In others we would compute in the cloud and just download final results for plotting. 
+Numerical model output often gets archived, distributed, and retrieved over the web. Some times, downloading data is most convenient. In other cases, it is preferable to compute in the cloud and just download final results. 
 
-`ClimateModels.jl` leverages standard Julia packages to read common file formats. [Downloads.jl](https://github.com/JuliaLang/Downloads.jl), [NetCDF.jl](https://github.com/JuliaGeo/NetCDF.jl), [DataFrames.jl](https://github.com/JuliaData/DataFrames.jl), [CSV.jl](https://github.com/JuliaData/CSV.jl), and [TOML.jl](https://github.com/JuliaLang/TOML.jl) are direct dependencies of `ClimateModels.jl`. This makes it easy to read and write such files.
+`ClimateModels.jl` has examples for most common file formats. These are handled via [Downloads.jl](https://github.com/JuliaLang/Downloads.jl), [NetCDF.jl](https://github.com/JuliaGeo/NetCDF.jl), [DataFrames.jl](https://github.com/JuliaData/DataFrames.jl), [CSV.jl](https://github.com/JuliaData/CSV.jl), and [TOML.jl](https://github.com/JuliaLang/TOML.jl).
 
 ```@example 1
 fil=joinpath(pathof(mc),"run02.csv")
@@ -198,4 +196,5 @@ CSV.File(fil) |> DataFrame
 summary(ans) # hide
 ```
 
-For examples with [NetCDF](https://github.com/JuliaGeo/NetCDF.jl) and [Zarr](https://github.com/meggart/Zarr.jl), please refer to [IPCC notebook](../examples/IPCC.html) (NetCDF) and [CMIP6 notebok](../examples/CMIP6.html) (Zarr).
+!!! note
+ For more examples with [NetCDF.jl](https://github.com/JuliaGeo/NetCDF.jl) and [Zarr.jl](https://github.com/meggart/Zarr.jl), please look at [IPCC notebook](../examples/IPCC.html) and [CMIP6 notebok](../examples/CMIP6.html).

docs/src/index.md

@@ -1,13 +1,10 @@
 # ClimateModels.jl
 
-This package provides a uniform interface to climate models of varying complexity and completeness. Models that range from low dimensional to whole Earth System models can be run and/or analyzed via this framework. 
-
-It also supports e.g. cloud computing workflows that start from previous model output available over the internet. Version control, using _git_, is included to allow for workflow documentation and reproducibility.
-
-The [JuliaCon 2021 Presentation](https://youtu.be/XR5hKCja0uw) provides a brief (8') overview and demo of the package.
-
-Please refer to [User Manual](@ref manual), [Examples](@ref examples), and [API Reference](@ref api) for more detail. 
-
+`ClimateModels.jl` provides a uniform interface to climate models of varying complexity and completeness. It is aimed at any model from highly simplified to whole Earth System models. 
+
+
+`ClimateModels.jl` also supports workflows that leverage model output available online. Version control, using _git_, allows for easy workflow documentation and reproducibility.
+
 ### [Table Of Contents](@id main-contents)
 
 ```@contents
@@ -16,12 +13,10 @@ Pages = [
  "examples.md",
  "API.md",
 ]
-Depth = 2
+Depth = 1
 ```
 
-- **JuliaCon 2021 Presentation**
- - [video recording (mp4)](https://youtu.be/XR5hKCja0uw)
- - [notebook view (html)](ClimateModelsJuliaCon2021.html)
- - [notebook source (jl)](https://github.com/gaelforget/ClimateModels.jl/blob/master/docs/src/ClimateModelsJuliaCon2021.jl)
+### Video 
 
-[![Screen Shot 2021-08-31 at 2 25 04 PM](https://user-images.githubusercontent.com/20276764/131556274-48f3df13-0608-4cd0-acf9-c3e29894a32c.png)](https://youtu.be/XR5hKCja0uw)
+- [JuliaCon 2021 Presentation](https://youtu.be/XR5hKCja0uw) (8') 
+- [JuliaCon 2023 Presentation](https://youtu.be/_Y6mNrN7eWA?si=gP_BITSTLHhFC20N) (20')

examples/FaIR_module.jl

@@ -32,7 +32,6 @@ Concrete type of `AbstractModelConfig` for `FaIR` model.
 Base.@kwdef struct FaIR_config <: AbstractModelConfig
  model :: String = "FaIR"
  configuration :: String = "rcp45"
- options :: OrderedDict{Any,Any} = OrderedDict{Any,Any}()
  inputs :: OrderedDict{Any,Any} = OrderedDict{Any,Any}()
  outputs :: OrderedDict{Any,Any} = OrderedDict(:C=>Float64[],:F=>Float64[],:T=>Float64[])
  status :: OrderedDict{Any,Any} = OrderedDict{Any,Any}()
@@ -95,4 +94,4 @@ function plot(scenarios,temperatures)
  f
 end
 
-end
+end

examples/Hector_module.jl

@@ -19,7 +19,6 @@ Concrete type of `AbstractModelConfig` for `Hector` model.
 Base.@kwdef struct Hector_config <: AbstractModelConfig
  model :: String = "Hector"
  configuration :: String = "hector_ssp245.ini"
- options :: OrderedDict{Any,Any} = OrderedDict{Any,Any}()
  inputs :: OrderedDict{Any,Any} = OrderedDict{Any,Any}()
  outputs :: OrderedDict{Any,Any} = OrderedDict{Any,Any}()
  status :: OrderedDict{Any,Any} = OrderedDict{Any,Any}()