Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

DQM output format and parsing script #147

Merged
merged 34 commits into from
Aug 23, 2024
Merged

Conversation

jlenain
Copy link
Collaborator

@jlenain jlenain commented Aug 2, 2024

This PR introduces a new format for the DQM output in FITS files, as well as a user script which, once a DQM analysis has been run on DIRAC, grabs the results from DIRAC, extracts the archive and ingests the results in the ZODB data base.

This is a fundamental brick towards an automatic pipeline which would launch the DQM systematically on DIRAC on data transfer from CEA to DIRAC, and results ingestion in ZODB.

This was tested to run fine locally (see http://lpnhe-cta2.in2p3.fr/nectarcam-dqm), but we need to merge this PR, to trigger the building of an Apptainer image in CI, which would then be used to launch a couple of DQM jobs on DIRAC, to be able to further test the user script from start to finish.

In case of issues along the way, another PR will be opened.

@jlenain jlenain added the enhancement New feature or request label Aug 2, 2024
@jlenain jlenain self-assigned this Aug 2, 2024
@jlenain jlenain requested a review from vmarandon August 2, 2024 20:37
Copy link

codecov bot commented Aug 2, 2024

Codecov Report

Attention: Patch coverage is 14.54545% with 47 lines in your changes missing coverage. Please review.

Project coverage is 48.94%. Comparing base (b306f30) to head (119b3dd).
Report is 23 commits behind head on main.

Files Patch % Lines
src/nectarchain/dqm/bokeh_app/main.py 0.00% 23 Missing ⚠️
src/nectarchain/dqm/dqm_summary_processor.py 16.66% 20 Missing ⚠️
src/nectarchain/dqm/start_dqm.py 0.00% 2 Missing ⚠️
src/nectarchain/dqm/trigger_statistics.py 0.00% 2 Missing ⚠️
Additional details and impacted files
@@            Coverage Diff             @@
##             main     #147      +/-   ##
==========================================
+ Coverage   40.76%   48.94%   +8.18%     
==========================================
  Files          65       64       -1     
  Lines        4452     4566     +114     
==========================================
+ Hits         1815     2235     +420     
+ Misses       2637     2331     -306     

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

@jlenain jlenain requested review from vmarandon and removed request for vmarandon August 23, 2024 11:59
@jlenain jlenain merged commit bf6b1f8 into cta-observatory:main Aug 23, 2024
10 of 11 checks passed
@jlenain jlenain deleted the scripts-dqm branch August 23, 2024 12:03
@jlenain jlenain mentioned this pull request Aug 23, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
enhancement New feature or request
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants