Merge pull request #2 from spaceml-org/new_release

New Release [WIP]

.github/workflows/install.yaml

@@ -0,0 +1,50 @@
+name: build
+
+env:
+  # Use the same ssh-agent socket value across all jobs
+  # Useful when a GH action is using SSH behind-the-scenes
+  SSH_AUTH_SOCK: /tmp/ssh_agent.sock
+
+on:
+  push:
+    branches: [ main ]
+  pull_request:
+    branches: [ '**' ]
+
+jobs:
+  build:
+
+    runs-on: ubuntu-latest
+    steps:
+      - name: Checkout repository
+        uses: actions/checkout@v3
+      # Start ssh-agent but set it to use the same ssh_auth_sock value.
+      # The agent will be running in all steps after this, so it
+      # should be one of the first.
+      - name: Set up Python 3.8
+        uses: actions/setup-python@v4 
+        with:
+          python-version: 3.8
+      - name: Install System Dependencies
+        run: |
+          sudo apt-get update
+          sudo apt-get install -y libfftw3-dev libopenblas-dev
+      - name: Install
+        run: |
+          python -m pip install --upgrade pip
+          pip install matplotlib
+          pip install jupyter
+          pip install numpy
+          pip install torch==1.11.0+cpu torchvision==0.12.0+cpu torchaudio==0.11.0+cpu -f https://download.pytorch.org/whl/cpu/torch_stable.html
+          pip install .[dev]
+          pip install sphinx
+          pip install sphinx-book-theme
+          pip install myst-nb
+          cd doc
+          make html linkcheck      
+      
+      - name: Upload to github pages 🚀
+        if: ${{ github.event_name == 'push' }}
+        uses: JamesIves/github-pages-deploy-action@v4
+        with:
+          folder: doc/_build/html # The folder the action should deploy.

README.md

@@ -1,66 +1,66 @@
-[![Install package and run tests](https://github.com/spaceml-org/karman/actions/workflows/install-and-test.yaml/badge.svg)](https://github.com/spaceml-org/karman/actions/workflows/install-and-test.yaml) 
-[![codecov](https://codecov.io/gh/Sceki/karman-private/branch/main/graph/badge.svg?token=XKMX6CPTNI)](https://codecov.io/gh/Sceki/karman-private)
-[![docs](https://readthedocs.org/projects/karman-project/badge/?version=latest)](https://karman-project.readthedocs.io/en/latest/?badge=latest)
-
+[![Install package and run tests](https://github.com/spaceml-org/karman/actions/workflows/install.yaml/badge.svg)](https://github.com/spaceml-org/karman/actions/workflows/install.yaml) 
 
+Karman - Data Driven Thermospheric Density Nowcast and Forecast with Machine Learning
 <!-- PROJECT LOGO -->
 <br />
 <p align="center">
-  <a href="https://github.com/spaceml-org/karman">
-    <img src="docs/source/__static/Karman_logo_black.png" alt="Logo" width="380" height="120">
+  <a href="https://spaceml-org.github.io/karman/">
+    <img src="docs/_static/Karman_logo_white.png" alt="Logo" width="380">
   </a>
   <p align="center">
-    Karman: Data Driven Thermospheric Density Modeling
+    Karman
     <br />
-    <a href="https://karman-project.readthedocs.io/en/latest/"><strong>Explore the docs »</strong></a>
+    <a href="https://spaceml-org.github.io/karman/"><strong>Explore the docs »</strong></a>
     <br />
     <br />
-    <a href="https://github.com/spaceml-org/karman/issues">Report Bug</a>
+    <a href="https://github.com/spaceml-org/karman/issues/new/choose">Report bug</a>
     ·
-    <a href="https://github.com/spaceml-org/karman/issues">Request Feature</a>
+    <a href="https://github.com/spaceml-org/karman/issues/new/choose">Request feature</a>
   </p>
 </p>
 
-Karman is a machine learning (ML) Python package for thermospheric density modelling. It was developed as a collaboration between the Heliophysics Division of NASA and [Trillium Technologies](https://trillium.tech/) under a NASA-funded Sole Source project. Its main goal is to improve our understanding and modeling of thermospheric density variations due to the Sun's influence. Furthermore, it also has the objective to provide a shared framework where people can design, train and benchmark their own thermospheric density models.
+Karman is a machine learning (ML) Python package for thermospheric density modelling. It was developed as a collaboration between the Heliophysics Division of NASA and [Trillium Technologies](https://trillium.tech/). 
+
+Its main goal is to improve our understanding and modeling of thermospheric density variations due to the Sun's influence. Furthermore, it also has the objective to provide a shared framework where people can design, train and benchmark their own thermospheric density models.
 
 ## Goals
 
-* Release of ML models to predict thermospheric density changes from solar and geomagnetic data, trained on precise orbit determination (POD)-derived thremospheric density measurements. 
-* A framework to support ingestion of solar irradiance, geomagnetic and thermospheric density data from different sources.
-* Benchmarking framework to enable comparison between ML and empirical models at different geomagnetic storm conditions, altitude, times, etc.
+* Release of ML models to nowcast & forecast thermospheric density changes from solar and geomagnetic data, trained on precise orbit determination (POD)-derived thremospheric density measurements. 
+* A framework to support ingestion of solar irradiance, geomagnetic and thermospheric density data from different sources. Currently, it supports data from: solar indexes (F10.7, M10.7, S10.7, Y10.7), geomagnetic indixes (Dst, Ap), SOHO EUV irradiance data, OMNIWeb high-resolution data, GOES EUV irradiance missions, POD-derived thermospheric density data.
+* Framework to enable comparison between ML and empirical models at different geomagnetic storm conditions, altitude, times, etc.
 
 ## Supported Data
 
 Currently, the Karman software supports several input data from different sources:
 * High frequency (i.e., every 30 or 10 seconds) POD-derived thermospheric density data from five space missions (CHAMP, GOCE, GRACE, SWARM-A, SWARM-B), from [TU Delft thermosphere data](http://thermosphere.tudelft.nl/).
-* NRLMSISE-00 and JB-08 thermospheric density empirical models inputs (and corresponding predicted density).
+* NRLMSISE-00 thermospheric density empirical models inputs (and corresponding predicted density).
 * Geomagnetic input data from [NASA's OMNIWeb high resolution data](https://omniweb.gsfc.nasa.gov/form/omni_min.html).
-* [EUV Stan Bands from FISM2](https://lasp.colorado.edu/lisird/data/fism_flare_bands/) empirical model (both daily and flare).
 * Solar irradiance proxies commonly used in empirical thermospheric density models (i.e., F10.7, S10.7, M10.7, Y10.7).
+* EUV Irradiance data from SOHO mission (from [LASP](https://lasp.colorado.edu/eve/data_access/eve_data/lasp_soho_sem_data/) website)
+* EUV Irradiance data from GOES mission (from [NOAA](https://www.ncei.noaa.gov/data/goes-space-environment-monitor/access/science/euvs/) website)
+
+We have dedicated tutorials and scripts to download and prepare each of these data sources for ML ingestion.
 
-![Karman Schematic Illustration](docs/source/__static/karman_schematic_illustration.png)
+![Karman Schematic Illustration](docs/_static/Karman_process.jpg)
 
 ## Installation, documentation, and examples
 
-https://karman-project.readthedocs.io/en/latest/
+https://spaceml-org.github.io/karman/
 
 ## Authors:
-* [Giacomo Acciarini](https://www.surrey.ac.uk/people/giacomo-acciarini)
-* [Edward Brown](https://www.cst.cam.ac.uk/people/ejeb4)
-* [Atılım Güneş Baydin](http://gbaydin.github.io/)
-* [Chris Bridges](https://www.surrey.ac.uk/people/chris-bridges)
+Currently active contributors are:
+* [Giacomo Acciarini](https://www.esa.int/gsp/ACT/team/giacomo_acciarini/)
+* [James Walsh](https://walsh.dev/)
 
-The project originated after the initial work of the [NASA-FDL 2021](https://frontierdevelopmentlab.org/fdl-2021) Solar Drag team, funded by the Heliophysics Division of NASA.
-
-Solar Drag team members: Researchers: Stefano Bonasera, Edward Brown, Jorge A. Pérez-Hernández, Bernard Benson; Faculty: Giacomo Acciarini, Atılım Güneş Baydin, Chris Bridges; Advisors: Eric Sutton, Moriba K. Jah.
+Many other people contributed to Karman since 2021 over the course of several FDLs. For a list of previous and current contributors see the "credits" page in the doc. 
 
 ## More info and how to cite:
 
 If you would like to learn more about or cite the techniques Karman uses, please see the following papers:
 
 * Giacomo Acciarini, Edward Brown, Thomas E. Berger, Madhulika Guhathakurta, James Parr, Christopher Bridges, Atılım Güneş Baydin. 2024. ["Improving Thermospheric Density Predictions in Low-Earth Orbit with Machine Learning"](https://agupubs.onlinelibrary.wiley.com/doi/pdfdirect/10.1029/2023SW003652). In Space Weather Journal.
 
-```
+```bibtex
 @article{acciarini2024improving,
   title={Improving Thermospheric Density Predictions in Low-Earth Orbit With Machine Learning},
   author={Acciarini, Giacomo and Brown, Edward and Berger, Tom and Guhathakurta, Madhulika and Parr, James and Bridges, Christopher and Baydin, At{\i}l{\i}m G{\"u}ne{\c{s}}},
@@ -75,7 +75,8 @@ If you would like to learn more about or cite the techniques Karman uses, please
 
 * Giacomo Acciarini, Edward Brown, Christopher Bridges, Atılım Güneş Baydin, Thomas E. Berger, Madhulika Guhathakurta. 2023. ["Karman - a Machine Learning Software Package for Benchmarking Thermospheric Density Models"](https://www.researchgate.net/publication/374005215_Karman_-a_Machine_Learning_Software_Package_for_Benchmarking_Thermospheric_Density_Models). In Advanced Maui Optical and Space Surveillance Technologies Conference (AMOS) – www.amostech.com.
 
-```
+
+```bibtex
 @inproceedings{karman-amos-acciarini,
   author    = {Giacomo Acciarini and Edward Brown and Christopher Bridges and Atılım Güneş Baydin and Thomas E. Berger and Madhulika Guhathakurta},
   title     = {Karman - a Machine Learning Software Package for Benchmarking Thermospheric Density Models},
@@ -86,9 +87,21 @@ If you would like to learn more about or cite the techniques Karman uses, please
 }
 ```
 
+* Shreshth Malik, James Walsh, Giacomo Acciarini, Thomas Berger, At{\i}l{\i}m G{\"u}ne{\c{s}}} Baydin. 2023. ["High-Cadence Thermospheric Density Estimation enabled by Machine Learning on Solar Imagery"](https://ml4physicalsciences.github.io/2023/files/NeurIPS_ML4PS_2023_237.pdf). In NeurIPS, Machine Learning and the Physical Sciences Workshop.
+
+```bibtex
+@article{malik2023high,
+  title={High-Cadence Thermospheric Density Estimation enabled by Machine Learning on Solar Imagery},
+  author={Malik, Shreshth A and Walsh, James and Acciarini, Giacomo and Berger, Thomas E and Baydin, At{\i}l{\i}m G{\"u}ne{\c{s}}},
+  booktitle={NeurIPS, Machine Learning and the Physical Sciences Workshop},
+  year={2023},
+  month={December}
+}
+```
+
 ## Acknowledgements:
 
-The authors would like to ackowledge the support and precious feedback of several people: [Dr. Madhulika Guhathakurta](https://science.nasa.gov/about-us/organization-and-leadership/lead-program-scientist-for-lws), [Tom Berger](https://www.colorado.edu/spaceweather/thomas-berger), and all the NASA-FDL 2021 reviewers.
+The authors would like to ackowledge the support and precious feedback of several people: [Dr. Madhulika Guhathakurta](https://science.nasa.gov/about-us/organization-and-leadership/lead-program-scientist-for-lws), and all the NASA-FDL 2021, 2023, 2024 reviewers.
 
 ## Contact:
 * `giacomo.acciarini@gmail.com`

docs/api.rst

@@ -0,0 +1,12 @@
+.. _api:
+
+API
+====
+
+Karman API
+
+.. autosummary::
+   :toctree: _autosummary
+   :recursive:
+
+   karman

docs/capabilities.ipynb

@@ -0,0 +1,22 @@
+{
+ "cells": [
+  {
+   "cell_type": "markdown",
+   "metadata": {},
+   "source": [
+    "# Capabilities\n",
+    "\n",
+    "Karman is an open-source package for data driven thermospheric density modeling with Machine Learning.\n",
+    "\n",
+    "It offers various functionalities to download and process input data from several sources, as well as machine learning models (also pre-trained) for thermospheric density nowcasting and forecasting."
+   ]
+  }
+ ],
+ "metadata": {
+  "language_info": {
+   "name": "python"
+  }
+ },
+ "nbformat": 4,
+ "nbformat_minor": 2
+}

docs/conf.py

@@ -0,0 +1,108 @@
+# Configuration file for the Sphinx documentation builder.
+#
+# This file only contains a selection of the most common options. For a full
+# list see the documentation:
+# https://www.sphinx-doc.org/en/master/usage/configuration.html
+
+# -- Path setup --------------------------------------------------------------
+
+# If extensions (or modules to document with autodoc) are in another directory,
+# add these directories to sys.path here. If the directory is relative to the
+# documentation root, use os.path.abspath to make it absolute, like shown here.
+#
+# import os
+# import sys
+# sys.path.insert(0, os.path.abspath('.'))
+
+
+# -- Project information -----------------------------------------------------
+
+project = "karman"
+copyright = "2022, 2023, 2024, 2025, Giacomo Acciarini, James Walsh"
+author = "Giacomo Acciarini, James Walsh"
+
+# The full version, including alpha/beta/rc tags
+import sys
+sys.path.append('../')
+import karman
+import os
+sys.path.insert(0, os.path.abspath('../karman'))
+
+release = karman.__version__
+
+
+# -- General configuration ---------------------------------------------------
+
+# Add any Sphinx extension module names here, as strings. They can be
+# extensions coming with Sphinx (named 'sphinx.ext.*') or your custom
+# ones.
+extensions = ["myst_nb", "sphinx.ext.autodoc", "sphinx.ext.doctest", "sphinx.ext.intersphinx", "sphinx.ext.autosummary","sphinx.ext.napoleon"]
+
+# build the templated autosummary files
+autosummary_generate = True
+autosummary_imported_members = True
+napoleon_google_docstring = True
+numpydoc_show_class_members = False
+panels_add_bootstrap_css = False
+
+autosectionlabel_prefix_document = True
+
+intersphinx_mapping = {
+    "numpy": ("https://numpy.org/doc/stable/", None),
+    "python": ("https://docs.python.org/3", None),
+    "matplotlib": ("https://matplotlib.org/stable/", None),
+}
+
+autoclass_content = 'both'
+
+# Add any paths that contain templates here, relative to this directory.
+templates_path = ["_templates",".DS_Store"]
+
+# List of patterns, relative to source directory, that match files and
+# directories to ignore when looking for source files.
+# This pattern also affects html_static_path and html_extra_path.
+exclude_patterns = ["_build", ".DS_Store"]
+
+
+# -- Options for HTML output -------------------------------------------------
+
+# The theme to use for HTML and HTML Help pages.  See the documentation for
+# a list of builtin themes.
+#
+html_theme = "sphinx_book_theme"
+
+# Add any paths that contain custom static files (such as style sheets) here,
+# relative to this directory. They are copied after the builtin static files,
+# so a file named "default.css" will overwrite the builtin "default.css".
+html_static_path = ["_static"]
+
+html_logo = "_static/Karman_logo_white.png"
+
+html_theme_options = {
+    "repository_url": "https://github.com/spaceml-org/karman",
+    "repository_branch": "main",
+    "path_to_docs": "docs",
+    "use_repository_button": True,
+    "use_issues_button": True,
+    "launch_buttons": {
+        "binderhub_url": "https://mybinder.org",
+        "notebook_interface": "jupyterlab"
+    },
+    "navigation_with_keys": False,
+}
+
+nb_execution_mode = "force"
+
+nb_execution_excludepatterns = [
+#    "target_notebook.ipynb",
+]
+
+latex_engine = "xelatex"
+
+myst_enable_extensions = [
+    "amsmath",
+    "colon_fence",
+    "deflist",
+    "dollarmath",
+    "html_image",
+]