-
Notifications
You must be signed in to change notification settings - Fork 350
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Change to frame finding in collated DQ script #4910
Conversation
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
LGTM - minor comments on whether 'old' usage should be kept as well
frames = gwdatafind.find_urls( | ||
args.ifo[0], | ||
frame_type, | ||
frames = find_frames( |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Is there a use-case for keeping the original method as well? I.e. setting frames
up according to the arguments.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I'm not immediately sue what the use case would be, but its simple enough to implement.
valid_bad_dq.protract(args.dq_padding) | ||
valid_bad_dq.coalesce() | ||
|
||
flagged_time += (valid_bad_dq & observing_flag).livetime |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
would valid_bad_dq ever not be during the observing flag??
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
This line of code is there to handle the possibility that padding the segment (the valid_bad_dq.protract call on line 117) causes the segment to overflow the edge of the observing segments.
Previously, the collated DQ script relied on gwdatafind to find frames for the past day. This was not appropriate for the MDC because gwdatafind would point at frames that did not actually match the MDC frames. This change allows a directory to be specified to read the frames from, intended to be the low-latency frame cache.
Standard information about the request
This is a: bug fix
This change affects the live search, only when using supervised fitting to produce stat files for the DQ statistic.
This change changes scientific output, but hopefully not by much.
Motivation
GWdatafind was pointing to frames that did not match the frames actually circulated in the MDC. This lead to a bug because the main PyCBC Live script and the dq fitting script did not agree on whether there were DQ flaged times, because they were getting their data from different sources.
Contents
Instead of relying on gwdatafind, specify a root directory that frames can be found in and a frame type in the supervised fitting config file. The structure of the frame directory is predictable, allowing the code to find the frames it needs.
Testing performed
Ran on the MDC analysis machines for days when the previous version of this code had crashed.