Make cluster abundance an independent module with a connector for firecrown

[m-aguena]

[marcpaterno]

The title of this issue does not make it clear to me what problem is being addressed. Can you please clarify the problem?

[m-aguena]

@marcpaterno the idea is to isolate the basic functionality for cluster abundance computation that is firecrown independent. Then we can have that as an isolated module (or eventually a library) that firecrown can import using a wrapper that makes the connection with firecrown specific objects (like updatables and parameters).
Maybe a better name would be: Isolate cluster theory functionality into a firecrown-independent module

[marcpaterno]

That helps, but it still seems to me that this is a proposed solution rather than a specification of a problem to be solved. What is the reason for (eventually) moving this code outside of the firecrown repository? The answer to that question will help determine the right way to go about doing it.

[mattkwiecien]

To summarize some of our discussion from last week. The cluster abundance calculations are being done in multiple different packages (Augur for forecasting, TJPCov for abundance covariance, Firecrown for the likelihood). To ensure consistency between these packages, we're trying to create some independent module that can be accessed by the other tools.

Because the abundance models are not purely theoretical predictions but also include phenomenological models, such as selection functions for our various cluster finders, CCL isn't quite a good fit for this code either.

@m-aguena we discussed taking the following approach, but still thinking through/prototyping:

• Creating a cluster likelihood factory function that builds the cluster theoretical model. This would be in the new external module.
• This factory function could then be used by Augur, TJPCov, Firecrown along with something similar to an .ini file that configures the theoretical model.
• ModelingTools in Firecrown could then load the cluster abundance likelihood from the external factory function (location TBD), and populate all models in firecrown with a consistent model / cosmology.

[mattkwiecien]

Some notes from our conversation today about design

• We want to start separating some cluster logic from statistics into models.
  • Models or Proxies? Let's define exactly what we want
  • What is the abstract interface for this?
  • Updatable solves the problem of needing custom arguments for different implementations, through yaml + ParamsMap
• If we proceed with Updatable implementation, this would be defined in our cluster models module. For now this "module" will just be a separate directory within Firecrown
• As with above, we still want to try to create a cluster likelihood factory function which produces the cluster abundance likelihood that lives on ModelingTools
  • ModelingTools is not set in stone, if we need to change this we can
  • We want to also support serialization of our cluster likelihood (to .ini for example)
• Need to remain cognizant of integration
  • Still need to better understand how things will work before we can nail down an integration approach for the theory vectors
  • Regardless we don't want to bog down the firecrown dev environment with packages that do very specific things
• There's a large PR coming into Firecrown that changes Updatable, so we want to make sure we use that version.

[mattkwiecien]

#345 implements the above.

During code review on this change set, the possibility of different data types for arguments for the integrand was brought up. Specifically, in our current implementation, the static type for mass_richness_proxy is just a float since we integrate it from lambda_min to lambda_max, for example. But this can be more complex and multidimensional.

It would be nice if we didn’t need to tie down the set of arguments required for the cluster abundance integrand. However, the current implementation does not support this. The workflow is:

flowchart TD;
    A[Integrator.integrate]--List of ints-->B[Integrator._integral_wrapper];
    B--mass, z, mass_proxy, z_proxy, etc -->C[AbundanceIntegrand];
    C--mass, z, mass_proxy, z_proxy, etc --> D[Kernel];

Loading

Which means all Kernels accept the same argument types. As you can see in the above workflow, this is ultimately determined by the _integral_wrapper inside the integrator, which turns the List<int> from the numerical integrator into named, typed arguments. This means if we add a Kernel that requires a more complex data type (e.g., mass_proxy: Tuple[float, float, float]), the change would need to be in the integrator and would affect all downstream data types.

Perhaps a simpler and more flexible approach would be to defer the responsibility of writing the mapping between the numerical integrator and integrand arguments to the user in their build_likelihood script. Then, we would pass these two methods, map() and integrand() down through our call stack. The user could still use all of the code we have written; they just need to construct their own Callable to be integrated using our pieces. Something like:

flowchart TD;
    A[build_likelihood]--map, integrand-->B[BinnedClusterNumberCounts];
    B--map, integrand -->C[BinnedClusterNumberCounts.compute_theory_vector];

Loading

I believe this approach would support any type of distribution in the integrand. Any thoughts? @vitenti @marcpaterno