Skip to content

Latest commit

 

History

History
191 lines (162 loc) · 14.2 KB

README.md

File metadata and controls

191 lines (162 loc) · 14.2 KB
Raw

Utilizing the 25m Dwingeloo Radio Telescope (DRT) to Study Fast Radio Bursts (FRBs) at ASTRON, the Netherlands 🇳🇱 📡

Advisors: Dr. Tammo Jan Dijkema & Co-advisor Proffesor Jason Hessels

This github repo is no longer active, for updates about the pipeline see the gitlab repo

"We look to the stars, but all we find is RFI." - Found on the back of a poster at ASTRON

Over summer 2024, I was a researcher at The Netherlands Institute for Radio Astronomy (ASTRON) with their Summer Research Programme. For my project, I was using and operating the 25-m Dwingeloo radio telescope. I am studing bright repeating Fast Radio Bursts (FRBs) to understand the potential connections between repeating and apparently non-repeating FRBs. As FRBs are hard to catch, I will also observe pulsars to both test the methodology and learn the relevant techniques.

This is me inside the control room, observing with the 25-m Dwingeloo Radio Telescope, and this is my confirmed detection of FRB20240619D.

Max Fine Observing with the 25-m Dwingeloo Radio Telescope

Pipeline GitLab Repo: Here

Big Picture Outline:

The Big Picture of my project is to learn & apply radio astronomy techniques for detecting FRBs. This entails understanding how the radio telescope turns radio signals into 'raw' data (in our case voltages), and then how the 'raw' data is further reduced by a data pipeline. The data pipeline searchs for and cuts out Radio Frequency Interference (RFI), and searchs for possible extragalactic radio signals, at present this is done with the presto package. Part of the Big Idea is to make a series of minor to moderate (or even major) improvements to the search pipeline. If I am very lucky, we will be able to detect a Repeating FRB over summer. However, in the more likely case we will be looking at a bright Pulsar to test as a final 'proof of concept' that the DRT can be used in dedicated campaigns to observe bright repeating FRBs.

Final Presentation:

YouTube Video

Table of Contents:

  • reading material
  • ref sheet of usefull cmds
  • software requirements and versions
  • example_pipeline_h5_output contains a notebook that opens a .h5 file, makes a waterfall plot for practice. Using a .h5 file made from the pipeline
  • frb_example_data_june_2024 contains a notebook making waterfall plots from CHIME .npy files, and two notebooks trying out fitburst on CHIME data, a simulation and a possible detection of an FRB
  • noise_channels contains several notebooks, and a script that looks at past observations taken and calculutes bad frequency channels to mask
  • inject_sims_into_fil, contains notebooks and scripts for injecting simulated bursts into .fil data for injection testing the pipeline
  • dbcan_clustering, contains a notebook for trying out dbscan for clustering candidates in 2d (DM, Time), contains rough code to edit check_frb.py to cluster
  • calling_fetch, contains a notebook and a few modified scripts from the gitlab repo. This enables fetch to be called from within check_frb.py, as well as moving the candidate .h5 files into good or bad directors and making dir plots for ones in the good dir
  • ddplan, contains a notebook trying out DDplan.py from presto on my laptop, and the new function that has been incorporated into the pipeline
  • presto_with_docker, contains a notebook for running, and executing presto commands - with presto running in a docker container
  • recording_baseband_data, contains a notebook for trying to record baseband data, take slices for good candidates, and slices for good candidates in other bands
  • Crab analysis, contains notebooks, and work for analysing the pulses from the crab
  • FRB analysis, contains a plotting notebook for my FRB detection. It also has python functions for dedispersing and bandwidth correction
  • Final Presentation Slides

Data Releases:

Observing (FRBs)

Details:

FRB20240619D:

  • See ATel #16690, detected by MeerKat and ATel #16732, detected by Westerbork
  • Expected DM: 465 (pc/cm^3)
  • Expected Ra, Dec: 297.37, -25.21 (deg)
  • We have limited observation time, typically its up ~23:00 - 04:00, I have to be physically in the telescope to take data...
  • I have made a postive detection! No Baseband data was recorded due to high RFI

FRB20240316A

FRB20240209A:

Suggestions For Future Work:

  • Have the camras mastodon bot post "good" candidates from fetch
    • Send as an email to myself, and Tammo, others as well
  • Add a poster outside of the Camras Telescope to include its FRB research (With a QR code to an engaging video!)
  • Try 3d DBscan clustering, taking into account the brightness of the detection (right now its time, and DM)
    • Probably not a substaintial improvement
  • Try out Transientx over PRESTO for the pipeline for speed
  • Have Desktop application to click to start program on Mercurius for ease of use
  • Auto stop when storage has less then 20Gb left
  • update FRB.cat file to be in .key format
  • Write a comprehensive observation log, including durations, headers, and clustering info
  • When there is a 'hyper-active' FRB, have a camras team try to take many hours of data - I will commute from Amsterdam for this.
    • Maybe make a post asking for extra volunteers
  • Inject Simulations into .fil files to test how many FRBs the pipeline recovers
    • Concern that bright FRB's might be marked as RFI, see unexpectedly low Crab pulses in the bottom right of this plot

TODO list:

Tasks:

Completed:

  • Understand how FRB signals from space turn into dynamic spectra. See flowchart.

  • Learn how to operate the 25-m Dwingeloo Radio Telescope.

  • Learn how to use Presto for single pulse searches and RFI removal.

  • Learn how the current pipeline works (check_frb.py). See flowchart.

    • Pipeline GitLab repo: here
    • My version of the pipeline: here
    • Create a file of known bad frequency channels to mask.
    • Modify start_frb.sh & check_frb.py scripts to load from a catalog file instead of hardcoded sources.

  • Learn how Fetch works and implement it into the pipeline.

    • Fetch is installed and working on Uranus!

  • Learn how TransientX works.

  • Understand what Burst Parameters can be observed & measured directly, and which ones can be inferred.

    • List out model components from FITBURST, polarization, fluence, etc.

  • Understand how to use fitburst.

  • Compare pipeline outputs when using the --ignorechan flag in check_frb.py

  • Try using fitburst on the CHIME data I have, simulated data, and my possible detection of FRB20240209A.

  • Try a clustering algorithm for reducing the total number of candidates (e.g. DBSCAN).

  • Implement dbscan clustering into check_frb.py

    • Reduces number of candidates by ~ the number of DM trials

  • Modify the scrits to record observations on Uranus (instead of Mercurius)

    • Paul, and Tammo did this, plugging in some cabels and running a data stream from mercurius to uranus
    • can record L and P bands directly onto Uranus

  • Modify check_frb.py to run fetch, and move the files into the good and bad dirs, make diagnostic .pngs

  • Add back log file for candidates

    • Added a #header

  • Added a log file for pipeline crashes

  • Added observing time, and number of good candidates to frb_dashboard.py

  • Work on making the pipeline run in real time - [x] Modify storage location of .fil files from /date to /date/>bandname< for runnning the pipeline in real time - [x] Modify frb_dashboard.py for the new file organization - [x] Make a good dir to hold all the good .h5 and .png files located at /date/good

  • Figure out what causes the pipeline to crash, and adjust the number of CPU cores as needed - [x] dashboard.py was using all the CPUS... fixed - [x] Limit GPU to one call of fetch - Good CPU limit seems around 12 per band?, No noticable increase after this - Still crashes, L1 and L2 recording seems unstable

    • Double Check if statments for --ignorechan option in check_frb.py
    • Look into making simulated, injecting simulations into .fil files for testing.
      • fitburst has a cool simulate_burst.py script that can simulate dedispersed or dispersed dynamic spectrums
      • will is a simulator that can be used to inject (and extract!) simulated pulses into .fil files!
        • Struggling on controlling the amplitude (SNR) of the injected signal, talked to Dirk who offered advice but haven't worked on this in a while

  • record raw voltages

    • Understand how to record raw voltages aka baseband data
    • Wrote function to slice a small chunk of data into a new file for a candidate
    • Write a function to convert arrival times for different frequency bands based on the DM
    • Write function to loop through candidates, and then delete the big baseband files
      • Implement into pipeline
    • Added storage on ram disk to frb_dashboard.py
    • Test
    • Read how the single_pulse_search works in presto, and how it determines the SNR - How it determines SNR - what time it uses for the time of the candidate (it is topocentric, top of band) - How many, and what size time boxcars in the search

  • Analysis of the Crab Pulsar (see Crab analysis:

    • Make plots of SNR, Fluence, $E_{\nu}$ for the crab over many hours of observations
    • Compare to 2019, 2024 Crab paper, and Nature Paper
    • Addtionaly make some stastical plots of the Crab in the different Bands

    To Do:

    • [ ]Implement TransientX into the pipeline. - Waiting for it to be installed on Uranus.

    • Write documenation for pipelines, new flowcharts

    • Understand what Paul and Tammo did to have the data record on Uranus

    • write an introduction section on FRBS, the DRT, and our observational parameters (bandwidth, devices, data points per second etc)

    • Fill in black boxes in the flowcharts.

      • Look into how presto actually removes RFI and finds pulses.
        • Look into how candidates are extracted from the .fil file.

    • Make a 'hip' mastodon bot to display the pngs from the good candidates

      • maybe make the pngs nan out detected RFI

    • Make a script to make .h5 files with full resolution for a good fetch candidate.

    • Injection testing the pipeline.

    • Test the pipeline on Crab or Pulsar and compare the number of recovered vs. missed bursts.

      • This would be a good test for our clustering methodology as well.

    • Combine data with other telescopes to measure fringes/localization.

    • Maybe make a docker container version of the pipeline.

    • Read more FRB papers.

    • Make nicely formatted documentation for the pipeline.

    • Understand what happens when intra-channel Dispersion is dominate.

    • Understand how our FRB search and CHIME's differ, IE with many antennas .

    • Make 'hip' popular science videos on research

    • Understand Red vs White Noise.

License

GPL-3.0