Sample scripts for validating future samples #1216

horohoo · 2023-10-06T16:40:26Z

I am updating ldmx-sw with sample validation scripts discussed in the 9/25/23 SWAN meeting. Here are the details.

Check List

I successfully compiled ldmx-sw with my developments
I ran my developments and the plots shown in 9/25/23 SWAN meeting show that they are successful.

I have included a sample configuration file for running the code (in .txt form that needs converted to .py, since GitHub doesn't support .py attachments to PRs):

val_config.txt

It can be executed with the syntax
ldmx fire val_config.py <input file to validate> <output file name, with .root extension>

bryngemark · 2023-10-09T08:36:19Z

hi @tylerhoroho-UVA , thanks for this! did you try using the validation module for plotting at all?

tomeichlersmith

I concur with @bryngemark , this is great and thank you for sharing :)

My comments are 100% cleaning comments.

Remove the undefined functions (I added some suggestions).
Format the code according to our style guide so its easier for other folks to collaborate.
Make the hard brem energy configurable? I'm not sure if this is desirable since we often set-it and forget-it, but its something to think about.

Formatting

In container images v4 and newer, clang-format is already installed so you can.

ldmx clang-format -i --style=Google DQM/include/DQM/SampleValidation.h DQM/src/DQM/SampleValidation.cxx
# commit these changes

DQM/src/DQM/SampleValidation.cxx

DQM/include/DQM/SampleValidation.h

horohoo · 2023-10-10T17:53:29Z

Hi @bryngemark, I made some effort to use the Validation module, but integration with this module would be more difficult than keeping everything contained in DQM. Since Einar developed the Validation module, maybe I could work with them to integrate the python code into Validation if that's where it should be kept?

bryngemark · 2023-10-11T07:58:30Z

Right, so what I think we ideally want is a combination of DQM for doing all the ldmx-sw processing/analyzing (like histogram booking and filling) and then a mirroring validation script that is just used to find those histograms, draw them, compare them, etc etc. So keeping what you have right now but just adding the Validation bit.

If you want inspiration you could look at Tom's recent PR here: #1215
I have a vision that we start using this combination to help streamline validation in the future. If it's too much work then just say no :)

horohoo · 2023-10-11T15:43:38Z

I can put in some more effort to integrating Validation into the workflow before deciding it's too much work. Thanks for linking Tom's PR; I think it will be helpful!

Co-authored-by: Tom Eichlersmith <31970302+tomeichlersmith@users.noreply.github.com>

bryngemark · 2023-11-15T18:56:48Z

DQM/src/DQM/SampleValidation.cxx

@@ -117,6 +121,8 @@ namespace dqm {
    if (pdgid > 2300) label = 16; //exotic (e.g., baryon with strangeness)


Have you considered what should happen if some PDGID doesn't match any of your defined criteria?

I have considered this and plan to make changes to account for this. The current setup was put together quickly so that Jessica could proceed with validation plots. I think I will fork "exotic" into "strange baryons" and "something else" (which would then require looking at the event level to see the mystery particle). I can also expand the A' label to be any dark mediator.

Sounds great. Just a catch-all in the end for anything unexpected can turn out to be very useful.

bryngemark · 2023-11-15T18:57:31Z

Validation/src/Validation/simparticles.py

+             density=True)
+
+    d.plot1d("SampleValidation/SampleValidation_pdgid_harddaughters" "PDG ID of hard primary daughter",
+             tick_labels=pdgid_labels,


This is really great. Makes it SO much easier to read and grasp the plot!

bryngemark · 2023-11-21T14:34:04Z

DQM/src/DQM/SampleValidation.cxx

@@ -89,7 +87,7 @@ namespace dqm {
  }

  int SampleValidation::pdgid_label(const int pdgid) {
-    int label = 0;
+    int label = 18;


Cool! Just add a comment that this is a catch-all/overflow bin for code readability

EinarElen · 2023-11-22T17:36:57Z

DQM/src/DQM/SampleValidation.cxx

+
+    if (pdgid == 3122 || pdgid == 3222 || pdgid == 3212 || pdgid == 3112 || pdgid == 3322 || pdgid == 3312) label = 16; // strange baryon
+
+    if (pdgid > 10000) label = 15; //nuclei


Is it worth differentiating between general nuclei and light ions here? The production of alpha, deuterium, and tritium from PN interactions is non-trivial.

I can include this distinction since I meant for the label to mainly catch remnants of a tungsten explosion. Do you think "light"/"heavy" nuclei is granular enough, with "light" being only hydrogen or helium nuclides?

The bertini cascade will attempt to take nucleons being ejected from the cascade that are "similar" and bunch them together into deuterium, tritium, alpha (called "Coalescence") so you can get these even for quite ordinary interactions.

There is a version of a test for what counts as a light ion here (that probably should live somewhere else...)
https://github.com/LDMX-Software/ldmx-sw/blob/8e6e9e195eb71ad78f7aa94648a179342a12e1f6/Biasing/include/Biasing/PhotoNuclearTopologyFilters.h#L62C1-L78

EinarElen · 2023-11-22T17:39:11Z

DQM/src/DQM/SampleValidation.cxx

+    double hard_thresh = 2500;
+
+    //Loop over all SimParticles
+    for (auto const& it : particle_map) {


Not something that needs to be fixed but something that is useful is that when iterating over a map you can do

for (auto const& [id, particle] : particle_map) { ... }

To define the id and particle variables directly without having to do it.second :)

EinarElen · 2023-11-22T17:39:50Z

DQM/src/DQM/SampleValidation.cxx

+
+    std::vector<int> primary_daughters;
+
+    double hard_thresh = 2500;


Why are we defining hard_thresh down here and redefining it later?

I needed to declare hard_thresh outside the first for loop since it is used in later loops. But in principle I should only declare it here without assigning it a value yet. This was likely a remnant from when hard_thresh was a hard-coded value. I can remove the value definition but will still keep the variable declared here.

EinarElen · 2023-11-22T17:40:32Z

Validation/src/Validation/simparticles.py

+             density=True)
+
+    d.plot1d("SampleValidation/SampleValidation_pdgid_harddaughters" "PDG ID of hard primary daughter",
+             tick_labels=pdgid_labels,


EinarElen

All good, assuming all of the changed files have been formatted before merged (they might already be)

bryngemark · 2023-11-29T19:55:00Z

@tomeichlersmith are the failed build tests a consequence of needing to rebase, or something else? if all is good then i think we can merge this

tomeichlersmith · 2023-11-29T19:58:46Z

The failed tests are due to UMN losing its network connection on Monday and thus the tests were unable to download an example CSV table condition.

  1:   Wget error 4 retreiving URL 'http://webusers.physics.umn.edu/~jmmans/ldmx/
  1:   condtest//testbeam22_pedestals.csv'

This is the third time this has happened... we may want to move our URL test somewhere else (or stop testing it since we don't use the internet retrieval of conditions right now).

EinarElen · 2023-11-29T20:02:22Z

I'm not super familiar with our test setup, but is there a way to mark that test as not fatal of it fails?

EinarElen · 2023-11-29T20:03:36Z

Is the plan to include this in some of the CI DQM configs btw?

jmmans · 2023-11-29T20:04:20Z

Dear all, I think a great solution (if possible) would be to put copies of the document in two locations (e.g. UMN and SLAC or Lund) and then fail only if /both/ fail. Jeremy

…

On 11/29/23 14:02, Einar Elén wrote: I'm not super familiar with our test setup, but is there a way to mark that test as not fatal of it fails? — Reply to this email directly, view it on GitHub <#1216 (comment)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/ABFUPYHBWUHHLGQNN5OLCELYG6IFVAVCNFSM6AAAAAA5WCOQ62VHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMYTQMZSGYYTSMRZGU>. You are receiving this because you are subscribed to this thread.Message ID: ***@***.***>

bryngemark · 2023-11-29T20:40:30Z

i think @EinarElen and @jmmans both make some excellent points here. we could punt on (as in go ahead and merge now) copying stuff into the CI DQM, maybe think a little more about what we'd like to include. copying the conditions to both slac and Lund could work.

horohoo requested review from tomeichlersmith, bryngemark, therwig and vdutta October 6, 2023 16:40

tomeichlersmith reviewed Oct 9, 2023

View reviewed changes

DQM/src/DQM/SampleValidation.cxx Outdated Show resolved Hide resolved

DQM/include/DQM/SampleValidation.h Outdated Show resolved Hide resolved

Horoho and others added 4 commits November 8, 2023 08:49

Sample scripts for validating future samples

043c3f1

Trimming undefined functions

91d70ed

Co-authored-by: Tom Eichlersmith <31970302+tomeichlersmith@users.noreply.github.com>

Trimming undefined functions

bd0ebe5

Co-authored-by: Tom Eichlersmith <31970302+tomeichlersmith@users.noreply.github.com>

A few changes that allow SampleValidation to work at 4 or 8 GeV

909d2b2

horohoo force-pushed the feature_SampleValidation branch from c5d06e8 to 909d2b2 Compare November 8, 2023 16:54

Horoho and others added 4 commits November 8, 2023 13:37

Changed threshold for 4 or 8 GeV hard brem selection

b407ba3

Adding ability to compare two different files

73497ed

Adding A' as tick mark in pdgid plot

c376dfa

Added missing comma in one plot

b3f0073

bryngemark reviewed Nov 15, 2023

View reviewed changes

Splitting strange baryons from exotics in PDG ID plot

7feeeff

bryngemark approved these changes Nov 21, 2023

View reviewed changes

Added comment on pdg id label for code readability

e141757

EinarElen reviewed Nov 22, 2023

View reviewed changes

Added distinction between light and heavy nuclei in PDG ID plots

26bbe9d

EinarElen approved these changes Nov 29, 2023

View reviewed changes

bryngemark approved these changes Nov 29, 2023

View reviewed changes

tomeichlersmith merged commit c060401 into trunk Nov 30, 2023
1 check passed

tomeichlersmith deleted the feature_SampleValidation branch November 30, 2023 13:56

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Sample scripts for validating future samples #1216

Sample scripts for validating future samples #1216

horohoo commented Oct 6, 2023

bryngemark commented Oct 9, 2023

tomeichlersmith left a comment

horohoo commented Oct 10, 2023

bryngemark commented Oct 11, 2023

horohoo commented Oct 11, 2023

bryngemark Nov 15, 2023

horohoo Nov 15, 2023

bryngemark Nov 15, 2023

bryngemark Nov 15, 2023

EinarElen Nov 22, 2023

bryngemark Nov 21, 2023

EinarElen Nov 22, 2023

horohoo Nov 27, 2023

EinarElen Nov 27, 2023

EinarElen Nov 22, 2023

EinarElen Nov 22, 2023

horohoo Nov 27, 2023

EinarElen Nov 22, 2023

EinarElen left a comment

bryngemark commented Nov 29, 2023

tomeichlersmith commented Nov 29, 2023

EinarElen commented Nov 29, 2023

EinarElen commented Nov 29, 2023

jmmans commented Nov 29, 2023 via email

bryngemark commented Nov 29, 2023

		@@ -117,6 +121,8 @@ namespace dqm {
		if (pdgid > 2300) label = 16; //exotic (e.g., baryon with strangeness)


		if (pdgid == 3122 \|\| pdgid == 3222 \|\| pdgid == 3212 \|\| pdgid == 3112 \|\| pdgid == 3322 \|\| pdgid == 3312) label = 16; // strange baryon

		if (pdgid > 10000) label = 15; //nuclei


		std::vector<int> primary_daughters;

		double hard_thresh = 2500;

Sample scripts for validating future samples #1216

Sample scripts for validating future samples #1216

Conversation

horohoo commented Oct 6, 2023

Check List

bryngemark commented Oct 9, 2023

tomeichlersmith left a comment

Choose a reason for hiding this comment

Formatting

horohoo commented Oct 10, 2023

bryngemark commented Oct 11, 2023

horohoo commented Oct 11, 2023

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

EinarElen left a comment

Choose a reason for hiding this comment

bryngemark commented Nov 29, 2023

tomeichlersmith commented Nov 29, 2023

EinarElen commented Nov 29, 2023

EinarElen commented Nov 29, 2023

jmmans commented Nov 29, 2023 via email

bryngemark commented Nov 29, 2023