Documentation, testing and CI (#2)

.github/workflows/generate_pdf.yaml

@@ -0,0 +1,25 @@
+on: push
+
+name: Compile JOSS paper
+
+jobs:
+  paper:
+    runs-on: ubuntu-latest
+    name: Paper Draft
+    steps:
+      - name: Checkout
+        uses: actions/checkout@v4
+      - name: Build draft PDF
+        uses: openjournals/openjournals-draft-action@master
+        with:
+          journal: joss
+          # This should be the path to the paper within your repo.
+          paper-path: docs/joss_paper/paper.md
+      - name: Upload
+        uses: actions/upload-artifact@v1
+        with:
+          name: paper
+          # This is the output path where Pandoc will write the compiled
+          # PDF. Note, this should be the same directory as the input
+          # paper.md
+          path: docs/joss_paper/paper.pdf

.github/workflows/test.yaml

@@ -0,0 +1,55 @@
+
+name: Unit and integration testing
+
+on:
+  push:
+    branches: [ "main", "test" ]
+  pull_request:
+    branches: [ "main" ]
+
+jobs:
+  build:
+    name: ${{ matrix.runner.name }} - Python ${{ matrix.python-version }}
+    runs-on: ${{ matrix.runner.os }}
+    strategy:
+      fail-fast: false
+      matrix:
+        python-version: 
+          - "3.9"
+          - "3.10"
+          - "3.11"
+        runner:
+          - name: Windows
+            os: windows-latest
+          - name: Ubuntu
+            os: ubuntu-latest
+    steps:
+      - uses: actions/checkout@v4
+      - name: Set up Python ${{ matrix.python-version }}
+        uses: actions/setup-python@v5
+        with:
+          python-version: ${{ matrix.python-version }}
+      - name: Install package
+        run: |
+          python -m pip install --upgrade pip
+          python -m pip install .
+      - name: Install test dependencies
+        run: |
+          python -m pip install .[test]
+      - name: Test pipx installation
+        run: |
+          python -m pip install pipx
+          pipx install .
+      - name: Lint with flake8
+        run: |
+          flake8 piglot test
+      - name: Test with pytest
+        run: |
+          pytest test/
+      - name: Install documentation dependencies
+        run: |
+          python -m pip install .[docs]
+      - name: Test building documentation
+        run: |
+          cd docs
+          make html

.gitignore

@@ -137,4 +137,7 @@ dmypy.json
 tmp/
 results/
 reference/
-.vscode/
+.vscode/
+
+# Example results
+examples/*/config/

README.md

@@ -1,8 +1,197 @@
-# piglot
-Parameter Identification by Global Optimisation Toolbox
+<div align="center">
+  <picture>
+    <source media="(prefers-color-scheme: dark)" srcset="docs/source/media/logo_dark.svg">
+    <source media="(prefers-color-scheme: light)" srcset="docs/source/media/logo.svg">
+    <img alt="Hashnode logo" src="docs/source/media/logo.svg" width="250">
+  </picture>
+</div>
 
-## Installation
-Clone the repository. Enter in the cloned directory and run
+A Python package for the optimisation of numerical responses.
+
+
+## Introduction
+
+Welcome to `piglot`, a Python tool taylored for the automated optimisation of responses from numerical solvers.
+We aim at providing a simple and user-friendly interface which is also easily extendable, allowing intergration with other solvers within the community.
+Whether you're working on structural analysis, material modelling, fluid dynamics, control systems or astrophysics (to name a few) using, for instance, finite element analysis, spectral methods or Monte Carlo methods, `piglot` provides a versatile solution for solving inverse problems.
+The primary emphasis is on derivative-free optimisation, ensuring compatibility with black-box solvers in scenarios where gradient information is not available, and cases where the function evaluations may be noisy.
+
+* **Integration with solvers:** We provide an extensible interface for coupling with physics solvers. As long as your solver can return a time-response for the fields you are interested, you can optimise it with `piglot`.
+* **Optimisation algorithms:** Off the shelf, there are several optimisers included in the package. Among them, we highlight our fully-fledged Bayesian optimisation (based on [BoTorch](https://botorch.org/)) that supports optimising stochastic and composite objectives and is highly customisable. Additional methods can also be easily implemented within `piglot`.
+* **Visualisation tools:** You can use the builtin tool `piglot-plot` to visualise the results of the optimisation. There are native plotting utilities for the optimised responses, the parameter history, objective history and, for supported solvers, live plotting of the currently running case. Also, an animation of the optimisation process can be exported.
+
+
+## Getting started
+
+We provide some examples to get you started with `piglot`.
+There are two modes of operation available: running using the given `piglot` and `piglot-plot` tools and configuration files, or building the optimisation problem in a Python script.
+
+### Using configuration files
+
+We use YAML configuration files to specify the optimisation problem to solve.
+This is the simplest form of using `piglot` and is the recommended approach unless you have a strong motive to use Python scripts (described [here](#using-python-scripts)).
+A simple analytical curve fitting problem is included to showcase how to use configuration files.
+In this case, we fit a quadratic expression of the type $f(x) = a x^2$.
+As a reference, a numerically generated reference from the expression $f(x) = 2 x^2$ is used (provided in the `examples/sample_curve_fitting/reference_curve.txt` file).
+We want to find the value for $a$ that better fits our reference (it should be 2).
+The configuration file for this example is:
+```yaml
+iters: 10
+
+optimiser: botorch
+
+parameters:
+  a: [1, 0, 4]
+
+objective:
+  name: fitting
+  solver:
+    name: curve
+    cases:
+      'case_1':
+        expression: <a> * x ** 2
+        parametric: x
+        bounds: [-5, 5]
+        points: 100
+  references:
+    'reference_curve.txt':
+      prediction: ['case_1']
+```
+You can find this file in `examples/sample_curve_fitting/config.yaml`
+We run 10 iterations using the `botorch` optimiser (our interface for Bayesian optimisation), and set the parameter `a` for optimisation with bounds `[0,4]` and initial value 1.
+Our optimisation objective is the fitting of an analytical curve, with the expression `<a> * x ** 2`.
+The notation `<a>` indicates that this parameter should be optimised.
+We also define a parameterisation using the variable $x$, where we sample the function between `[-5,5]` with 100 points.
+Finally, we compare this generated response (with the label `case_1`) with our reference, given from the file `reference_curve.txt`
+
+To run this example, open a terminal inside the `piglot` repository, enter the `examples/sample_curve_fitting` directory and run piglot with the given configuration file
+```bash
+cd examples/sample_curve_fitting
+piglot config.yaml
 ```
-pip3 install -e .[full]
+You should see an output similar to
+```
+BoTorch: 100%|██████████████████████████████████████████████████████| 10/10 [00:00<00:00, 17.66it/s, Loss: 8.8505e-08]
+Completed 10 iterations in 0.56614s
+Best loss:  8.85050592e-08
+Best parameters
+- a:     1.999508
+```
+As you can see, piglot correctly identifies the `a` parameter close to the expected value of 2, and the error of the fitting is in the order of $10^{-8}$.
+In addition to these outputs, `piglot` creates an output directory, with the same name of the configuration file (minus the extension), where it stores the optimisation data.
+
+To visualise the optimisation results, use the `piglot-plot` utility.
+In the same directory, run
+```bash
+piglot-plot best config.yaml
 ```
+Which will display the best observed value for the optimisation problem.
+You should see the following output in the terminal
+```
+Best run:
+Start Time /s    0.587397
+Run Time /s      0.004439
+a                1.999508
+Name: 18, dtype: object
+Hash: 2313718f75bc0445aa71df7d6d4e50ba82ad593d65f3762efdcbed01af338e30
+Objective:  8.85050592e-08
+```
+The script will also plot the best observed response, and its comparison with the reference response: 
+![Best case plot](docs/source/simple_example/best.svg)
+
+Now, try running (this may take some time)
+```bash
+piglot-plot animation config.yaml
+```
+This generates an animation for all the function evaluations that have been made throughout the optimisation procedure.
+You can find the `.gif` file(s) inside the output directory, which should give something like:
+![Best case plot](docs/source/simple_example/animation.gif)
+
+
+### Using Python scripts
+
+Another way of using `piglot` is via its package and Python modules.
+This approach may offer increase flexibility in the setup of the optimisation problem, at the cost of increased complexity and verbosity.
+A sample script equivalent to the configuration file for the problem described in [the previous section](#using-configuration-files) is provided in `examples/sample_curve_fitting/config.py`, given by:
+```python
+import os
+import shutil
+from piglot.parameter import ParameterSet
+from piglot.solver.solver import Case
+from piglot.solver.curve.solver import CurveSolver
+from piglot.solver.curve.fields import CurveInputData, Curve
+from piglot.objectives.fitting import Reference, MSE
+from piglot.objectives.fitting import FittingObjective, FittingSolver
+from piglot.optimisers.botorch.bayes import BayesianBoTorch
+
+# Set up output and temporary directories
+output_dir = 'config'
+tmp_dir = os.path.join(output_dir, 'tmp')
+if os.path.isdir(output_dir):
+    shutil.rmtree(output_dir)
+os.makedirs(output_dir, exist_ok=True)
+
+# Set up optimisation parameters
+parameters = ParameterSet()
+parameters.add('a', 1.0, 0.0, 4.0)
+
+# Set up the reference
+reference = Reference('reference_curve.txt', ['case_1'], output_dir)
+
+# Set up the solver to use
+input_data = CurveInputData('case_1', '<a> * x ** 2', 'x', (-5.0, 5.0), 100)
+case_1 = Case(input_data, {'case_1': Curve()})
+solver = CurveSolver([case_1], parameters, output_dir, tmp_dir=tmp_dir)
+
+# Set up the fitting objective
+references = {reference: ['case_1']}
+fitting_solver = FittingSolver(solver, references)
+objective = FittingObjective(parameters, fitting_solver, output_dir, MSE())
+
+# Set up the optimiser and run optimisation
+optimiser = BayesianBoTorch(objective)
+value, params = optimiser.optimise(10, parameters, output_dir)
+print(f"Optimal value: {value}")
+print(f"Optimal parameters: {params}")
+```
+Run with
+```bash
+python config.py
+```
+Example output
+```
+BoTorch: 100%|██████████████████████████████████████████████████████| 10/10 [00:00<00:00, 16.75it/s, Loss: 8.9167e-08]
+Completed 10 iterations in 0.59692s
+Best loss:  8.91673999e-08
+Best parameters
+- a:     1.999506
+Optimal value: 8.916739991036405e-08
+Optimal parameters: [1.99950592]
+```
+
+
+### Installation
+
+You can install `piglot` by only installing the main scripts or as a standard Python package.
+If you only intend to use the `piglot` and `piglot-plot` binaries, we strongly recommend the first option, as it avoids having to manage the dependencies in your Python environment.
+However, if you wish to use the tools in the `piglot` package, you may have to resort to the second option.
+Currently, we only support Python 3.9 onwards.
+
+#### Option 1: Install binaries
+
+This option is recomended for end-users that only need to interact with the provided `piglot` and `piglot-plot` scripts.
+We use [`pipx`](https://github.com/pypa/pipx) to install the package in an isolated environment with the required dependencies (we recommend reading the pipx documentation to check the advantages of using this approach).
+  1. Install `pipx` in your system using the instructions [here](https://github.com/pypa/pipx#install-pipx)
+  2. Clone the `piglot` repository.
+  3. Open your favourite terminal, change directory to the recently cloned `piglot` repository and run: `pipx install .`
+  4. Confirm the package is correctly installed by calling the `piglot` and `piglot-plot` executables.
+
+
+#### Option 2: Install package
+
+We recommend this option for users aiming to use the `piglot` package directly.
+Note that this option also provides the `piglot` and `piglot-plot` scripts, but requires manually handling the installation environment.
+  1. Clone the `piglot` repository.
+  2. Open your favourite terminal, change directory to the recently cloned `piglot` repository and run: `pip install .`
+  3. Confirm the package is correctly installed by calling the `piglot` and `piglot-plot` executables.
+

docs/joss_paper/paper.md

@@ -0,0 +1,84 @@
+---
+title: 'piglot: an Open-source Package for Derivative-free Optimisation of Numerical Responses'
+tags:
+  - Python
+  - computational mechanics
+  - inverse problems
+  - derivative-free optimisation
+  - Bayesian optimisation
+  - parameter identification
+authors:
+  - name: R. P. Cardoso Coelho
+    orcid: 0000-0001-9989-964X
+    affiliation: "1, 2"
+  - name: A. Francisca Carvalho Alves
+    orcid: 0000-0003-1214-5453
+    affiliation: "1, 2"
+  - name: T. M. Nogueira Pires
+    orcid: 0009-0000-1518-2845
+    affiliation: "1"
+  - name: F. M. Andrade Pires
+    orcid: 0000-0002-4802-6360
+    corresponding: true
+    affiliation: "1, 2"
+affiliations:
+ - name: Faculty of Engineering, University of Porto, Porto, Portugal
+   index: 1
+ - name: Institute of Science and Innovation in Mechanical and Industrial Engineering, Porto, Portugal
+   index: 2
+date: 7 February 2024
+bibliography: references.bib
+
+---
+
+# Summary
+`piglot` is an open-source Python tool taylored for the automated optimisation of responses stemming from numerical solvers. With this tool we aim at providing a simple and user-friendly interface which is also easily extendable, allowing intergration with other solvers within the community. `piglot` provides a versatile solution for solving inverse problems on several research areas, such as structural analysis, material modelling, fluid dynamics, control systems or astrophysics, using, for instance, finite element analysis, spectral methods or Monte Carlo methods. The primary emphasis is on derivative-free optimisation, ensuring compatibility with black-box solvers in scenarios where gradient information is not available, and cases where the function evaluations may be noisy.
+
+![Logo of `piglot`. \label{fig:piglot_logo}](../source/media/logo.svg){width=35%}
+
+# Statement of need
+
+The increasingly growing interest in computational analysis for engineering problems has been driving the development of more accurate, robust and efficient methods and models.
+With the advent of this technology, the application of the so-called inverse problems has been gaining traction over   the last years, where one seeks optimised parameters, geometries, configurations or models for numerical problems arising in engineering.
+In this context, in the past years, some packages have been developed to automate the identification of parameters [@nevergrad,@optuna_2019].
+These packages, however, do not provide an interface for different solvers, so the optimisation of numerical responses is not readily available.
+
+In this work, we present `piglot` an open-source Python package for automated optimisation of numerical responses, such as responses stemming from finite element simulations.
+In particular, focus is placed on derivative-free optimisation, to allow compatibility with black-solvers where gradient information may be unavailable.
+In this context an extensible interface for coupling with physics solvers is provided.
+As long as the solver can return a time-response for the fields of interest, it is possible to optimise it with `piglot`.
+Notwithstanding, some solvers are already provided, namely a solver for fitting analytical functions, a solver for the in-house finite element code `Links`, a solver for the finite element software `Abaqus`, and a solver for the clustering-based reduced-order model `CRATE` package [@Ferreira2023].
+
+For the optimisation itself, several optimisation methods are implemented and available, such as DIRECT, LIPO, Bayesian optimisation, among others.
+Particularly, a significant effort has been employed into Bayesian optimisation algorithms, backed with an open-source implementation [@balandatBoTorchFrameworkEfficient2020] and allowing for single- and multi-objective optimisation of both noise-free and stochastic objectives.
+Furthermore, a novel composite Bayesian optimisation strategy is available for curve-fitting problems, which, in our tests, severely outperforms classical optimisation approaches [@Coelho2023optm].
+
+The package also provides a builtin tool `piglot-plot` to visualise the results of the optimisation.
+There are native plotting utilities for the optimised responses, the parameter history, objective history and, for supported solvers, live plotting of the currently running case.
+The package also includes full documentation for a clear installation and usage, supporting a simple framework for new developments. 
+With this in mind, a thorough automated testing is incorporated, ensuring the compliance of new developments.
+
+In \autoref{fig:piglot_example} a scheme of the workflow of `piglot` is illustrated.
+There are two modes of initialisation available: using `.yaml` configuration files, or building the optimisation problem in a Python script. 
+The use of configuration files is the simplest and recommended approach of using `piglot`.
+During the optimisation there is a continuous exchange of information between the physics solvers, `piglot`, and the optimisers.
+Whereas the optimisers are responsible for providing a candidate solution for the parameters, $\boldsymbol{\theta}$, based on the loss function value, $J(\boldsymbol{\theta})$, the physics solvers receive the parameters, $\boldsymbol{\theta}$, and compute the numerical response, $\boldsymbol{\sigma}$, accordingly.
+The results of the optimisation can be then visualise using the `piglot-plot` tool.
+
+
+![Schematic illustration of `piglot`. \label{fig:piglot_example}](piglot.svg){width=100%}
+
+
+The `piglot` package has been successfully used for the identification of constitutive parameters for classical elasto-plastic models from multi-scale simulations, crystal plasticity models with mechanically-induced martensitic transformations [@cardosocoelhoMultiscaleModelCombining2023] and models for amorphous polymers [@ALVES2023112488].
+Moreover, this tool has also demonstrated its potential in the material design of different microstructures, such as particulate PC/ABS polymer blends.
+
+
+With this package, we aim to provide a simple and effective tool for general optimisation of numerical responses, which can be easily extended for other solvers in the community.
+
+
+# Acknowledgements
+
+R. P. Cardoso Coelho and A. Francisca Carvalho Alves gratefully acknowledge the support provided by Fundação para a Ciência e a Tecnologia (FCT) through the scholarships with references 2020.07159.BD and 2020.07279.BD, respectively.
+This research has also been supported by Instituto de Ciência e Inovação em Engenharia Mecânica e Engenharia Industrial (INEGI).
+
+# References