Change to frame finding in collated DQ script #4910
Merged
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| frames = gwdatafind.find_urls( | ||
| args.ifo[0], | ||
| frame_type, | ||
| frames = find_frames( |
There was a problem hiding this comment.
Is there a use-case for keeping the original method as well? I.e. setting frames up according to the arguments.
There was a problem hiding this comment.
I'm not immediately sue what the use case would be, but its simple enough to implement.
| valid_bad_dq.protract(args.dq_padding) | ||
| valid_bad_dq.coalesce() | ||
|
|
||
| flagged_time += (valid_bad_dq & observing_flag).livetime |
There was a problem hiding this comment.
would valid_bad_dq ever not be during the observing flag??
There was a problem hiding this comment.
This line of code is there to handle the possibility that padding the segment (the valid_bad_dq.protract call on line 117) causes the segment to overflow the edge of the observing segments.
GarethCabournDavies
approved these changes
Oct 18, 2024
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Previously, the collated DQ script relied on gwdatafind to find frames for the past day. This was not appropriate for the MDC because gwdatafind would point at frames that did not actually match the MDC frames. This change allows a directory to be specified to read the frames from, intended to be the low-latency frame cache.
Standard information about the request
This is a: bug fix
This change affects the live search, only when using supervised fitting to produce stat files for the DQ statistic.
This change changes scientific output, but hopefully not by much.
Motivation
GWdatafind was pointing to frames that did not match the frames actually circulated in the MDC. This lead to a bug because the main PyCBC Live script and the dq fitting script did not agree on whether there were DQ flaged times, because they were getting their data from different sources.
Contents
Instead of relying on gwdatafind, specify a root directory that frames can be found in and a frame type in the supervised fitting config file. The structure of the frame directory is predictable, allowing the code to find the frames it needs.
Testing performed
Ran on the MDC analysis machines for days when the previous version of this code had crashed.