Patching to ECAL lin-reg tracking

[tvami]

I am updating ldmx-sw, here are the details.

What are the issues that this addresses?

This resolves #1405
Several approaches were tried, in the end I'm cleaning up the input to the matrix to be decomposed, and making sure that the determinant is non-zero.
While digging into this I denote many potential improvements in the code.

Check List

• I successfully compiled ldmx-sw with my developments
• I ran my developments and the following shows that they are successful.

[tvami]

OK, I'll move this back to "draft", the signal (right) looks way to much like background (left) now.
So I'll have to look into this in more details

[tvami]

I'm getting there, still not the best

[tomeichlersmith]

How are you defining best? If you are cutting out singular matrices, could there be some lin-reg tracks "found" that were just "found" due to numerical instability of a singular matrix? I could see this effect being very distinct in the signal case where the ECal as a whole is fairly quiet.

[tvami]

Yes but we have more tracks now in signal! (blue is new red is old, right?)
So the changes seeing here are a bit counter-intuitive, how does removing singular matrices lead to more tracks?
Also I want to measure how it affect the other masses, I have a student Jihoon looking into that part now.

[tomeichlersmith]

ah yes, you're right - that is suspicious.

[tvami]

Gosh... I'm ready for review, but there is waaay to many things to improve on this part of the code. My approach is to make something minimally sensible now, and add // TODO to the point that needs to be improved. This can be a project for an undergrad (I'm sure we'll find somebody at UCSB).

As for the physics in the version pushed now:

Looks pretty good: we loose tracks (that were anyway nonsensical [hits from the same layer, exactly the same hits, inverting singular matrices....]), but we still have move tracks in the background case.

[tvami]

This can be a project for an undergrad

I'll make a reminder issue about this after we merged this

[tomeichlersmith]

Looks good to me, I just have a cleanup 🧹 and a technical question 🤔

[tvami]

Funny thing is that the KS score is not big enough for the signal to even flag the change:

[tomeichlersmith]

Yea, KS testing is somewhat weighted by the bin content and so the fact that 0 lin-reg tracks are very close for a vast majority of events means that the KS test value will be above our arbitrary threshold.

(In a perfect world, we would do a full comparison of the events with something like root-diff and then make histograms if any change cropped up there. The limiting thing in this case is storing the golden event files or implementing the complex workflow to re-generate them everytime.)

Edit: The N Lin-Reg Tracks distribution is also closer in the PR validation sample. Not sure why this is but could be from using different samples.

[tvami]

The N Lin-Reg Tracks distribution is also closer in the PR validation sample.

Ahh no sorry, I think I introduced a mistake in the last push regarding the initialization of the r2

[tvami]

The N Lin-Reg Tracks distribution is also closer in the PR validation sample.

Ahh no sorry, I think I introduced a mistake in the last push regarding the initialization of the r2

False alarm
https://github.com/LDMX-Software/ldmx-sw/blob/117acb437eabca116030524eee056eefe1385228/Ecal/src/Ecal/EcalVetoProcessor.cxx#L1075C15-L1079
the requiremenet on r_corr > .6 makes it ok