Skip to content

Commit

Permalink
default
Browse files Browse the repository at this point in the history
  • Loading branch information
Jorgedavyd committed Jan 3, 2025
1 parent 8e751da commit 29080d2
Show file tree
Hide file tree
Showing 5 changed files with 28 additions and 27 deletions.
18 changes: 10 additions & 8 deletions paper/agujournaltemplate.aux
Original file line number Diff line number Diff line change
Expand Up @@ -2,13 +2,15 @@
\bibstyle{apacite}
\@writefile{toc}{\contentsline {section}{\numberline {1}Introduction}{2}{}\protected@file@percent }
\@writefile{toc}{\contentsline {section}{\numberline {2}LASCO Calibration Routines}{2}{}\protected@file@percent }
\@writefile{toc}{\contentsline {section}{\numberline {3}L-Multilayered UNet-like Partial Convolution Network}{3}{}\protected@file@percent }
\@writefile{toc}{\contentsline {subsection}{\numberline {3.1}Partial Convolutions}{3}{}\protected@file@percent }
\@writefile{toc}{\contentsline {subsection}{\numberline {3.2}Loss function}{3}{}\protected@file@percent }
\@writefile{toc}{\contentsline {section}{\numberline {4}Data}{4}{}\protected@file@percent }
\citation{morrill2006calibration}
\@writefile{toc}{\contentsline {section}{\numberline {3}Fuzzy image reconstruction routines}{3}{}\protected@file@percent }
\@writefile{toc}{\contentsline {section}{\numberline {4}L-Multilayered UNet-like Partial Convolution Network}{3}{}\protected@file@percent }
\@writefile{toc}{\contentsline {subsection}{\numberline {4.1}Partial Convolutions}{3}{}\protected@file@percent }
\@writefile{toc}{\contentsline {subsection}{\numberline {4.2}Loss function}{3}{}\protected@file@percent }
\newlabel{tab: example}{{1}{5}{}{}{}}
\@writefile{toc}{\contentsline {section}{\numberline {5}Training}{5}{}\protected@file@percent }
\@writefile{toc}{\contentsline {section}{\numberline {6}Results and Discussion}{5}{}\protected@file@percent }
\@writefile{toc}{\contentsline {section}{\numberline {7}Conclusions and future work}{5}{}\protected@file@percent }
\@writefile{toc}{\contentsline {section}{\numberline {8}Open Research}{6}{}\protected@file@percent }
\@writefile{toc}{\contentsline {section}{\numberline {5}Data}{5}{}\protected@file@percent }
\@writefile{toc}{\contentsline {section}{\numberline {6}Training}{5}{}\protected@file@percent }
\@writefile{toc}{\contentsline {section}{\numberline {7}Results}{5}{}\protected@file@percent }
\@writefile{toc}{\contentsline {section}{\numberline {8}Conclusions and future work}{5}{}\protected@file@percent }
\@writefile{toc}{\contentsline {section}{\numberline {9}Open Research}{6}{}\protected@file@percent }
\gdef \@abspage@last{6}
14 changes: 7 additions & 7 deletions paper/agujournaltemplate.fdb_latexmk
Original file line number Diff line number Diff line change
@@ -1,5 +1,5 @@
# Fdb version 4
["pdflatex"] 1731683146.3169 "agujournaltemplate.tex" "agujournaltemplate.pdf" "agujournaltemplate" 1731683146.66491 2
["pdflatex"] 1735910625.20657 "agujournaltemplate.tex" "agujournaltemplate.pdf" "agujournaltemplate" 1735910625.5816 2
"/usr/share/texmf-dist/fonts/map/fontname/texfonts.map" 1727349926 3524 cb3e574dea2d1052e39280babc910dc8 ""
"/usr/share/texmf-dist/fonts/tfm/jknappen/ec/ectt1000.tfm" 1727349926 1536 06717a2b50de47d4087ac0e6cd759455 ""
"/usr/share/texmf-dist/fonts/tfm/public/amsfonts/cmextra/cmbsy5.tfm" 1727349926 1120 1e8878807317373affa7f7bba4cf2f6a ""
Expand Down Expand Up @@ -88,12 +88,12 @@
"/usr/share/texmf-dist/tex/latex/url/url.sty" 1727349926 12796 8edb7d69a20b857904dd0ea757c14ec9 ""
"/usr/share/texmf-dist/tex/latex/xcolor/xcolor.sty" 1727349926 55487 80a65caedd3722f4c20a14a69e785d8f ""
"/usr/share/texmf-dist/web2c/texmf.cnf" 1727349926 41588 b43d3e860a4f94167ee1e725ff526a72 ""
"/var/lib/texmf/fonts/map/pdftex/updmap/pdftex.map" 1730886873.20255 5312047 b07fcd2a9090df96fc745b92a3db793b ""
"/var/lib/texmf/web2c/pdftex/pdflatex.fmt" 1730886860 7112992 df1b11a02f5c3ca462c6cbb5ab44f133 ""
"agujournal2019.cls" 1731367162 40688 b442a0d4e8a299883855595037ff6651 ""
"agujournaltemplate.aux" 1731683146.60324 1176 cc8d5671406d2e473c4040645671c316 "pdflatex"
"agujournaltemplate.tex" 1731683145.89658 13822 e2654476056d73422bf881ceda2de39f ""
"trackchanges.sty" 1731367162 26287 09689e5e9db973d136f0537b8570fadf ""
"/var/lib/texmf/fonts/map/pdftex/updmap/pdftex.map" 1734717382.80671 5312047 b07fcd2a9090df96fc745b92a3db793b ""
"/var/lib/texmf/web2c/pdftex/pdflatex.fmt" 1734717306 7113009 9d6c81c9cd29e128a762e63430df19ba ""
"agujournal2019.cls" 1731690459.7938 40688 b442a0d4e8a299883855595037ff6651 ""
"agujournaltemplate.aux" 1735910625.51692 1318 cfd5a7b0aca804cf46f9f3850a71dfb8 "pdflatex"
"agujournaltemplate.tex" 1735910625.12692 14141 57b4f6bb05855c3c3c8a398b892b4bdc ""
"trackchanges.sty" 1731690459.80047 26287 09689e5e9db973d136f0537b8570fadf ""
(generated)
"agujournaltemplate.aux"
"agujournaltemplate.log"
Expand Down
Binary file modified paper/agujournaltemplate.pdf
Binary file not shown.
Binary file modified paper/agujournaltemplate.synctex.gz
Binary file not shown.
23 changes: 11 additions & 12 deletions paper/agujournaltemplate.tex
Original file line number Diff line number Diff line change
Expand Up @@ -22,23 +22,19 @@
\end{keypoints}

\begin{abstract}
CorKit is an open-source deep learning framework designed for the calibration and restoration of Large Angle and Spectrometric Coronagraph (LASCO) imagery, providing an accessible alternative to proprietary software like IDL. This framework integrates modern deep learning techniques, including an L-multilayered UNet-like partial convolution network, to enhance image reconstruction and address limitations in traditional calibration methods. By democratizing access to high-quality calibrated coronagraph data, CorKit facilitates advanced solar and space physics research, particularly in studying Coronal Mass Ejections (CMEs). The proposed framework redefines LASCO calibration, replacing fuzzy logic with data-driven image restoration, ensuring accuracy and adaptability for contemporary research needs.
\end{abstract}

\section{Introduction}

A coronagraph is an optical instrument designed to block the direct incidence of light from an object, typically a star. It constitutes one of the most relevant source of information to study phenoma related to the emission of radiation overall. For instance, Coronal Mass Ejections (CMEs), a solar outburst of ionized particles into the interestelar medium, can be sighted through the usage of these instruments.
A coronagraph is an optical instrument that blocks direct starlight, particularly useful for studying stellar radiation phenomena. One key application is observing Coronal Mass Ejections (CMEs) - powerful eruptions of ionized particles from the Sun into interstellar space. These observations have significantly advanced our understanding of solar dynamics and outburst mechanisms.
The Large Angle and Spectrometric Coronagraph (LASCO) aboard SOHO, a joint ESA-NASA mission launched in 1995, has been instrumental in detecting potentially hazardous CMEs that can disrupt Earth's geomagnetic field. SOHO studies the Sun from its core through the outer corona and solar wind.

Thanks to the imagery received from coronagraphs, our knowledge about solar phenomena and the dynamics of several types of solar outbursts has been cleared out throughout the years.
Space instrument data follows a hierarchical processing system: Level 0 (raw telemetry), Level 1 (calibrated physical units), and Level 2+ (derived products). While raw data is typically stored in restricted scientific databases, calibration to Level 1 traditionally relies on IDL's SolarSoftware library using the `reduce\_level\_1.pro` routine.

SOHO, a joint project of ESA and NASA, was launched in 1995. It is designed to study the Sun from its core to the outer corona and the solar wind. The Large Angle and Spectrometric Corongraph (LASCO) is one of the instruments on SOHO; it observes the solar corona through the coronagraph ideation of light blockage. It is a fundamental tool to detect hazardous CMEs that can alter our geomagnetic field.
This dependency on proprietary IDL software impedes open science in astrophysics. Additionally, the coronagraph calibration process, developed around 2000, uses outdated techniques like 32x32 block reconstruction with fuzzy recompositors. Modern deep learning architectures have demonstrated superior image reconstruction capabilities.

Spacecrafts instruments' data products are ordered by level of processing: Level 0 (Raw data and telemetry), Level 1 (Data Calibrated in physical units), and Level 2,3,... (Further feature engineered data products for diverse purposes). Generally, the non-calibrated products (Level 0 or intermediate representations) are stored in large databases, with personalized access for scientific research. The SolarSoftware library of IDL is used to calibrate the raw data into Level 1 using the `reduce_level_1.pro` routine.

This is a well established tool for scientifc computing, yet the access to it is constrained by the programming language licensing and usage requirements. As long as IDL is licensed, open science for astrophysics is not possible. Therefore, the development of an open-source alternative would be beneficial for the scientific comunity.

On the other hand, the calibration process of coronagraphs were adapted to the computing and reconstruction knowledge known by 2000. Data loss, tipically on 32 by 32 blocks, were reconstructed with fuzzy recompositors to recreated the dynamics around the missing block. However, modern deep learning architectures are more suitable for image reconstruction empirically.

That's why, Corkit was created with the purpose of democritizing the access to high-quality calibrated products for scientific analysis of corongraph data and to redefine the calibration steps to fit modern practices.
Corkit was developed to address these limitations by democratizing access to calibrated coronagraph data and modernizing the calibration pipeline using current best practices.

\section{LASCO Calibration Routines}

Expand All @@ -61,15 +57,18 @@ \section{LASCO Calibration Routines}
\begin{enumerate}
\item Subtract bias.
\item Divide by the corrected exposure factor.
\item \textbf{Deep learning aided recontruction. (Both C2 and C3)}
\item Multiply by calibration factor.
\item Multiply by (inverse) vignetting function/array.
\item Subtract stray light. (Just C3)
\item Distortion correction.
\item Multiply by (distortion corrected) mask.
\item Rectify image to solar north up.
\item \textbf{Deep learning aided recontruction. (Both C2 and C3)}
\end{enumerate}

\section{Fuzzy image reconstruction routines}
If an image has more than 0 and less than 100 missing blocks, the Fuzzy logic routine substracts a background image from the unprocessed level-0 or level-0.5 image to account for the gradient due to the F-corona, then, if there's less than 100 32 by 32 missing blocks within the image, the fuzzy logic extrapolation is applied to fill the missing data \cite{morrill2006calibration}.

\section{L-Multilayered UNet-like Partial Convolution Network}

\subsection{Partial Convolutions}
Expand Down Expand Up @@ -149,7 +148,7 @@ \section{Data}
\section{Training}
This model, as seen in Table \ref{tab: example}, was pre-trained using Adam optimizer, with a learning rate of 0.0002 (lr) and gradient clip at 0.005 (gc) for 20 hours with a Nvidia RTX 4070 dropping the physical constraints from the loss function. Then fine-tuned with a learning rate of 0.00005 and gradient clip of 0.0005 including the physical constraint terms.

\section{Results and Discussion}
\section{Results}


\section{Conclusions and future work}
Expand Down

0 comments on commit 29080d2

Please sign in to comment.