Improved numcosmo connector (#369)

- Removed unnecessary models and NumCosmo run scripts - Introduced new NumCosmo app (0.21.1) with required features - Updated example to remove deprecated variable usage - Removed unused variables from example values.ini - Updated NumCosmo connector to utilize new facilities (0.21.0) for serialization - Updated example ini to use proper Cosmosis configuration - Reorganized examples testing in CI - Added power spectra to NumCosmo conversions - Moved conversion functions to NamedParameters, added unit tests - Updated unit tests for new NumCosmo connector - Conducted more testing for next connector helper functions - Inverted conversion order since bool is int - Tested set/get in NumCosmo likelihoods - Tested empty NumCosmo data objects - Employed cleaner parametrization (omega_b, omega_c) instead of (omega_m, omega_c) - Created new NumCosmo cookbook - Resolved tabs issue in codecov.yml, changed tabs to spaces, removed comments, and linked to documentation - Upgraded codecov-action to v4, added flag to fail CI if upload fails We are passsing this review even though Coveralls is complaining because the changed lines have 100% test coverage, as verified by CodeCov.
LSSTDESC · Feb 6, 2024 · 0c4ae2d · 0c4ae2d
1 parent e95c803
commit 0c4ae2d
Show file tree

Hide file tree

Showing 31 changed files with 1,078 additions and 1,161 deletions.
diff --git a/.github/workflows/ci.yml b/.github/workflows/ci.yml
@@ -86,9 +86,9 @@ jobs:
       - name: Running mypy
         shell: bash -l {0}
         run: |
-          mypy -p firecrown -vv
-          mypy -p examples -vv
-          mypy -p tests -vv
+          mypy -p firecrown
+          mypy -p examples
+          mypy -p tests
       - name: Running pylint
         shell: bash -l {0}
         run: |
@@ -98,64 +98,76 @@ jobs:
       - name: Running pytest
         shell: bash -l {0}
         run: python -m pytest -vv --runslow --cov firecrown --cov-report xml
-      - name: Running example - cosmosis - cosmic-shear
+      - name: Running example - cosmic-shear - cosmosis
         shell: bash -l {0}
         run: |
           cd examples/cosmicshear
           python generate_cosmicshear_data.py
           cosmosis cosmicshear.ini
-      - name: Running example - cosmosis - des-y1-3x2pt
+      - name: Running example - cosmic-shear - NumCosmo
         shell: bash -l {0}
         run: |
-          cd examples/des_y1_3x2pt
-          cosmosis des_y1_3x2pt.ini
-      - name: Running example - cosmosis - des-y1-3x2pt-PT
+          cd examples/cosmicshear
+          numcosmo from-cosmosis cosmicshear.ini --matter-ps eisenstein_hu --nonlin-matter-ps halofit 
+          numcosmo run test cosmicshear.yaml
+      - name: Running example - des-y1-3x2pt - cosmosis
         shell: bash -l {0}
         run: |
           cd examples/des_y1_3x2pt
+          cosmosis des_y1_3x2pt.ini
           cosmosis des_y1_3x2pt_PT.ini
-      - name: Running example - cobaya - des-y1-3x2pt
+      - name: Running example - des-y1-3x2pt - cobaya
         shell: bash -l {0}
         run: |
           cd examples/des_y1_3x2pt
           cobaya-run cobaya_evaluate.yaml
-      - name: Running example - cobaya - des-y1-3x2pt-PT
-        shell: bash -l {0}
-        run: |
-          cd examples/des_y1_3x2pt
           cobaya-run cobaya_evaluate_PT.yaml
-      - name: Running example - NumCosmo - des-y1-3x2pt
+      - name: Running example - des-y1-3x2pt - NumCosmo
         shell: bash -l {0}
         run: |
           cd examples/des_y1_3x2pt
-          python numcosmo_run.py
-      - name: Running example - NumCosmo - des-y1-3x2pt-PT
-        shell: bash -l {0}
-        run: |
-          cd examples/des_y1_3x2pt
-          python numcosmo_run_PT.py
-      - name: Running example - cosmosis - srd_sn
+          numcosmo from-cosmosis des_y1_3x2pt.ini --matter-ps eisenstein_hu --nonlin-matter-ps halofit 
+          numcosmo run test des_y1_3x2pt.yaml
+          numcosmo from-cosmosis des_y1_3x2pt_PT.ini --matter-ps eisenstein_hu --nonlin-matter-ps halofit 
+          numcosmo run test des_y1_3x2pt_PT.yaml
+      - name: Running example - srd_sn - cosmosis
         shell: bash -l {0}
         run: |
           cd examples/srd_sn
           cosmosis sn_srd.ini
-      - name: Running example - NumCosmo - srd_sn
+          cosmosis sn_only.ini
+      - name: Running example - srd_sn - NumCosmo
         shell: bash -l {0}
         run: |
           cd examples/srd_sn
-          python numcosmo_run.py
-      - name: Running example - cosmosis - cluster_number_counts
+          numcosmo from-cosmosis sn_srd.ini
+          numcosmo run test sn_srd.yaml
+          numcosmo from-cosmosis sn_only.ini
+          numcosmo run test sn_only.yaml
+      - name: Running example - cluster_number_counts - cosmosis
         shell: bash -l {0}
-        run: |
+        run: |  
           cd examples/cluster_number_counts
           python generate_rich_mean_mass_sacc_data.py
           cosmosis cluster_counts_redshift_richness.ini
           cosmosis cluster_mean_mass_redshift_richness.ini
           cosmosis cluster_counts_mean_mass_redshift_richness.ini
+      - name: Running example - cluster_number_counts - NumCosmo
+        shell: bash -l {0}
+        run: |
+          cd examples/cluster_number_counts
+          numcosmo from-cosmosis cluster_counts_redshift_richness.ini --matter-ps eisenstein_hu
+          numcosmo run test cluster_counts_redshift_richness.yaml
+          numcosmo from-cosmosis cluster_mean_mass_redshift_richness.ini --matter-ps eisenstein_hu
+          numcosmo run test cluster_mean_mass_redshift_richness.yaml
+          numcosmo from-cosmosis cluster_counts_mean_mass_redshift_richness.ini --matter-ps eisenstein_hu
+          numcosmo run test cluster_counts_mean_mass_redshift_richness.yaml
       
       - name: Upload coverage reports to Codecov
         if: ${{ (matrix.os == 'ubuntu') && (matrix.python-version == '3.11') }}
-        uses: codecov/codecov-action@v3
+        uses: codecov/codecov-action@v4
+        with:
+          fail_ci_if_error: true
         env:
           CODECOV_TOKEN: ${{ secrets.CODECOV_TOKEN }}
 

diff --git a/codecov.yml b/codecov.yml
@@ -1,67 +1,38 @@
-# comment: false
+# See https://docs.codecov.com/docs/codecovyml-reference for config reference
+# To validate: `curl -X POST --data-binary @codecov.yml https://codecov.io/validate`
+
 comment:
   layout: "diff, flags, files"
   behavior: default
-  # if true: only post the comment if coverage changes
   require_changes: false  
-
-  # [true :: must have a base report to post]
   require_base: false        
-
-  # [true :: must have a head report to post]
   require_head: true       
-
-  # [true :: only show coverage on the git diff]
   hide_project_coverage: false 
 
 coverage:
-
-  # What precision do you want the coverage value to be
   precision: 1
-  # Which direction to you want to round the coverage value
   round: down
-  # The value range where you want the value to be green
-  range: 95..100
+  # red below 70, yellow between 70 and 95, green above 95.
+  range: 70..95 
 
   status:
-    # The codecov/patch status only measures lines adjusted 
-    # in the pull request or single commit, if the commit is 
-    # not in a pull request. This status provides an indication 
-    # on how well the pull request is tested.
     patch:
       default:
-        # basic
         target: 100%
         threshold: 0%
-
-        # advanced
-        branches: 
-          - master
-        if_ci_failed: error #success, failure, error, ignore
+        branches: [master]
+        if_ci_failed: error
         only_pulls: false
 
-    # The codecov/project status measures overall project 
-    # coverage and compares it against the base of the pull 
-    # request or parent commit.
     project:
       default: 
-        # auto will use the coverage from the base commit 
-        # (pull request base or parent commit) coverage to 
-        # compare against.
         target: auto
-        # Allow the coverage to drop by X%, and posting a success status
         threshold: 0%
-
-        # advanced settings
-        branches: 
-          - master
-        if_ci_failed: error #success, failure, error, ignore
+        branches: [master]
+        if_ci_failed: error
         only_pulls: false
-        # These behaviors allows users to specify what should happen 
-        # to the project status in case changes causes overall coverage 
-        # to drop, but are part of good development practices.
-        removed_code_behavior: fully_covered_patch #off, removals_only, adjust_base",
+        removed_code_behavior: fully_covered_patch
 
 github_checks:
-	annotations: true #,false
+  annotations: true
 
diff --git a/docs/environment.yml b/docs/environment.yml
@@ -10,7 +10,7 @@ dependencies:
   - flake8
   - jupyter
   - mypy
-  - numcosmo>=0.18.2
+  - numcosmo>=0.21.1
   - numpy>=1.21.0
   - plantuml
   - pip>=20.1  # pip is needed as dependency

diff --git a/docs/index.rst b/docs/index.rst
@@ -28,8 +28,6 @@ It can also be used as a library in other contexts, and so the installation of F
    Installation Quickstart<install_quick.rst>
    Using Firecrown<basic_usage.rst>
 
-
-
 .. toctree::
    :maxdepth: 1
    :caption: Developing with Firecrown
@@ -39,6 +37,13 @@ It can also be used as a library in other contexts, and so the installation of F
    Developer Notes<dev-notes.rst>
    Contributing<contrib.rst>
 
+.. toctree::
+   :maxdepth: 2
+   :caption: Cookbook
+   :name: cookbook
+
+   NumCosmo <numcosmo_cookbook.rst>
+
 .. toctree::
    :maxdepth: 1
    :caption: Reference

diff --git a/docs/numcosmo_bestfit.rst b/docs/numcosmo_bestfit.rst
@@ -0,0 +1,81 @@
+=======================================
+NumCosmo: Firecrown Likelihood Best Fit
+=======================================
+
+This recipe outlines the process to compute the best fit for Supernova SNIa data using
+the Firecrown example with NumCosmo. Ensure that you have already installed 
+Firecrown, CosmoSIS, and NumCosmo and set the environment variable ``FIRECROWN_DIR``
+to the location of the Firecrown installation.
+
+.. _convert-configuration-file-sn-srd:
+
+Convert Configuration File
+==========================
+
+Use the NumCosmo command-line tool to convert the ``sn_srd.ini`` file into a NumCosmo
+configuration in a ``yaml`` file. Since this likelihood does not require a 
+power-spectrum likelihood, you can use the ``from-cosmosis`` command without any extra
+flags `--matter-ps eisenstein_hu` nor `--nonlin-matter-ps halofit`. To mute the output 
+of the consistency cosmosis module, use ``--mute-cosmosis``.
+
+.. code-block:: bash
+
+   cd $FIRECROWN_DIR/examples/srd_sn
+   numcosmo from-cosmosis sn_srd.ini --mute-cosmosis
+
+This command creates a file called ``sn_srd.yaml``, NumCosmo's experiment file. It 
+contains cosmological and likelihood parameters, modeling choices, and chosen precision.
+
+.. _compute-bestfit-sn-srd:
+
+Run NumCosmo App
+================
+
+Execute the NumCosmo app with the experiment file to compute the best fit. The 
+following command saves the output in a new ``yaml`` file. Use the ``--help`` flag for 
+available options.
+
+.. code-block:: bash
+
+   cd $FIRECROWN_DIR/examples/srd_sn
+   numcosmo run fit sn_srd.yaml --output sn_srd-bestfit.yaml
+
+The best fit is saved in ``sn_srd-bestfit.yaml``. Inspect the file to view the 
+best-fitting values. This file serves as the starting point for subsequent runs.
+
+Restart Minimization Algorithm (Optional)
+=========================================
+
+If dealing with a high-dimensional parameter space or non-converging algorithms,
+consider using the `--restart` flag. This restarts the minimization algorithm from the
+best-fit found so far.
+
+.. code-block:: bash
+
+   cd $FIRECROWN_DIR/examples/srd_sn
+   numcosmo run fit sn_srd.yaml --output sn_srd-bestfit.yaml --starting-point sn_srd-bestfit.yaml --restart 1.0e-3 0.0
+
+The command restarts the minimization algorithm if the absolute change in the 
+likelihood is less than 1.0e-3 and the relative change is less than 0.0.
+
+Note: ``sn_srd-bestfit.yaml`` cannot be used as the experiment file for the 
+``numcosmo`` command-line tool since it lacks data and model information.
+
+Results Summary
+===============
+
+* **Starting Point:**
+
+  * m2lnL: 1215
+  * Parameters:
+
+    * sn_ddf_sample_M: -19.30
+    * Omegac: 0.260
+
+* **Best Fit:**
+
+  * m2lnL: 1.70934
+  * Parameters:
+
+    * sn_ddf_sample_M: -19.426
+    * Omegac: 0.265
diff --git a/docs/numcosmo_cookbook.rst b/docs/numcosmo_cookbook.rst
@@ -0,0 +1,16 @@
+=================================================
+Cookbook using NumCosmo
+=================================================
+
+Recipes for using Firecrown through the NumCosmo connector for cosmological analyses.
+
+.. toctree::
+   :maxdepth: 2
+   :caption: NumCosmo Cookbook
+   :name: numcosmo_cookbook
+
+   Computing Best-Fit <numcosmo_bestfit.rst>
+   Computing Fisher Matrix <numcosmo_fisher.rst>
+   Computing Fisher Bias <numcosmo_fisher_bias.rst>
+
+