Skip to content

Commit

Permalink
Add validation installation to the just commands and rename the dir (#…
Browse files Browse the repository at this point in the history
…1441)

I am updating _ldmx-sw_, here are the details.

### What are the issues that this addresses?
Resolves #1439
```
just install-validation
```
installs Validation module
and then
```
cd Validation/src
denv python3 -m Validate -h
```
works, and then 
```
denv python3 -m Validate compareDir/ --systems ecal.shower_feats
```
makes nice plots.

The rename allows now to run this at any directory. The change in _differ_ is to make sure we dont have a circular definition of matplotlib
  • Loading branch information
tvami authored Sep 6, 2024
1 parent 0018079 commit 6e8949c
Show file tree
Hide file tree
Showing 14 changed files with 26 additions and 33 deletions.
39 changes: 10 additions & 29 deletions Validation/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -5,25 +5,13 @@ Python package focused on comparing two or more "similar" LDMX event or histogra
## Installation
One may use this python module either inside or outside the ldmx-sw development container.

### Inside `ldmx`
Since the environment inside of the container is a bit more restrictive, we need to
specifically tell `pip` some extra information that it cannot deduce on its own.
### Inside `ldmx-sw`
Run the dedicated `just` command to install
```
ldmx python3 -m pip install Validation/ \
--upgrade \
--target install/python/ \
--no-cache
just install-validation
```
`python3 -m pip install Validation/` is the standard `pip` install method.
We add `--upgrade` to tell `pip` it should overwrite the package if it already has been
installed before which is helpful in the case where someone is updating the code and running
the new code within the container. The `--target install/python/` arguments tell `pip`
where to install the package. This directory is where we currently store our python modules
and is where the container expects them to be. The `--no-cache` argument tells `pip` to
not use a cache for downloading any dependencies from the internet which is necessary since
`pip` will not be able to write to the cache location within the container.

### Outside `ldmx`
### Outside `ldmx-sw`
Outside container it is helpful to put the Validation module inside a virtual environment.
This makes it easier to keep track of what you are doing and isolate the dependencies of `Validation`
from other python packages that may be on your system.
Expand All @@ -44,39 +32,32 @@ Using Validation without a virtual environment and outside of the container is n


## Usage
This module _cannot_ be run from ldmx-sw directory.
`import Validation` prefers the local directory instead of the installed path so it tries to
load from the `ldmx-sw/Validation` directory.

We could fix this by renaming the package inside Validation or renaming the directory.

### CLI
The Validation module is constructed to do some common tasks quickly on the command line.
Printing out its help message shows how to run it and gives you the details on what
parameters to provide.
```
python3 -m Validation -h
denv python3 -m Validate -h
```
which should be run with `ldmx` if the module was installed in the container.

For example if you would like to compare the ECAL shower features, put the input histograms into a directory,
e.g. `compareDir` with names that are separated with an underscore, e.g. `histo_new.root` and `histo_ref.root`,
then run the following command.
```
ldmx python3 -m Validation compareDir/ --systems ecal.shower_feats
denv python3 -m Validate compareDir/ --systems ecal.shower_feats
```
This will produce plots in the `compareDir` directory and will include "new" and "ref" in the plot legend.

### In Script
Similar to the CLI, you can develop your own python script using Validation.
Simply `import Validation` where you want to be using it.
Similar to the CLI, you can develop your own python script using Validate.
Simply `import Validate` where you want to be using it.
**Remember**: The plotting functions assume the user is in an interactive notebook
unless the `out_dir` parameter is provided.

### In Notebook
Again, accessing this module post-installation is the same as other modules `import Validation`.
Again, accessing this module post-installation is the same as other modules `import Validate`.
This can help you develop plots that don't come pre-made within the Validation module.
**If you are developing Validation and testing within a notebook**, you will need to reboot
**If you are developing Validate and testing within a notebook**, you will need to reboot
the python kernel anytime you wish to test changes to the Validation module. This is necessary
because Jupyter keeps modules cached in memory during normal running in order to save time
when re-executing cells with `import` statements.
File renamed without changes.
File renamed without changes.
Original file line number Diff line number Diff line change
Expand Up @@ -5,7 +5,6 @@
import re

# external dependencies
import matplotlib.pyplot as plt
import matplotlib
import uproot
# us
Expand Down Expand Up @@ -81,7 +80,7 @@ def plot1d(self, column, xlabel,
grp_name from the constructor.
If out_dir is not provided, we assume we are in a notebook and
simply `plt.show()` the figure. If out_dir is not None (i.e. it
simply `matplotlib.pyplot.show()` the figure. If out_dir is not None (i.e. it
was defined), we assume we are in a non-interactive script and
write the figure to a PDF in the output file and then clear
the figure.
Expand All @@ -107,7 +106,7 @@ def plot1d(self, column, xlabel,
hist_kwargs : dict
All other key-word arguments are passed into each File.plot1d
"""
fig = plt.figure('differ',figsize=(11,8))
fig = matplotlib.pyplot.figure('differ',figsize=(11,8))
raw_ax, ratio_ax = fig.subplots(
nrows = 2,
sharex = 'col',
Expand Down Expand Up @@ -151,7 +150,7 @@ def plot1d(self, column, xlabel,
ratio_ax.tick_params(axis='x', rotation=90)

if out_dir is None :
plt.show()
matplotlib.pyplot.show()
else :
if file_name is None :
if isinstance(column, str) :
Expand Down
File renamed without changes.
File renamed without changes.
File renamed without changes.
File renamed without changes.
File renamed without changes.
File renamed without changes.
File renamed without changes.
File renamed without changes.
File renamed without changes.
13 changes: 13 additions & 0 deletions justfile
Original file line number Diff line number Diff line change
Expand Up @@ -174,3 +174,16 @@ compile ncpu=num_cpus() *CONFIG='': (configure CONFIG) (build ncpu)

# re-build ldmx-sw and then run a config
recompFire config_py *ARGS: build (fire config_py ARGS)

# install the validation module
# `python3 -m pip install Validation/` is the standard `pip` install method.
# We add `--upgrade` to tell `pip` it should overwrite the package if it already has been
# # installed before which is helpful in the case where someone is updating the code and running
# # the new code within the container. The `--target install/python/` arguments tell `pip`
# # where to install the package. This directory is where we currently store our python modules
# # and is where the container expects them to be. The `--no-cache` argument tells `pip` to
# # not use a cache for downloading any dependencies from the internet which is necessary since
# # `pip` will not be able to write to the cache location within the container.
# # install the python Validation plotting module
install-validation:
denv python3 -m pip install Validation/ --upgrade --target install/python/ --no-cache

0 comments on commit 6e8949c

Please sign in to comment.