Inferential Statistics and Information Theoretical Measures: An Approach to Interference Detection in Radio Astronomy [thesis]

Morgan R Dameron
In a time when technology is rapidly growing, radio observatories are now able to expand their computational power to achieve higher receiver sensitivity power and a more flexible realtime computing approach to probe the universe for its composition and study new astronomical phenomena. This allows searches to go deeper into the universe, and results in the recording of massive quantities of observed data. At the same time, this increases the amount of radio frequency interference (RFI) found
more » ... the obtained observatory data. The high power of RFI easily masks the low power of extraterrestrial signals, making them hard to detect and potentially blinds radio telescopes from parts of the universe. Between the influx of RFI and the massive amounts of data recorded per second, a need for robust, efficient, high-performance, real-time RFI characterization and flagging algorithms has become abundantly clear. While there are current RFI characterization and flagging algorithms already implemented specifically for Radio Astronomy, many are not openly published and are fine tuned to the specific application a radio telescope is being used for, i.e hydrogen mapping, "fast radio burst" (FRB) detection, or transient and pulsar searches. Often the current RFI detection algorithms are implemented after post-processing has been completed. This means that data can be lost which could increase RFI signal strength while further decreasing the signal strength of celestial signals, making them harder to detect. We work with the "rawest" form of observatory data engineers have access to, complex-valued channelized voltages and their respective high resolution power spectral densities. As a baseline or "ground truth" we use Median Absolute Deviation (MAD) to detect RFI in complex-valued channelized voltages and Spectral Kurtosis (SK) to detect RFI in power spectral density data. In a new perspective, we use inferential statistics and information theoretical measures to formulate new real-time RFI detection algorithms for both multi-domain (time-frequency) data formats. Using inferential statistics, we apply the Shapiro-Wilk Test for Normality on complex-valued channelized voltages. We then apply information theoretical measures by extracting the Differential Spectral Entropy and Spectral Relative Entropy of both complex-channelized voltages and power spectral density data. The baseline RFI excision algorithms are compared against our novel algorithms to determine how effective and robust the new interference detection algorithms are. The completion of this work would not have been possible if I did not have the encouragement, support, and feedback from all of my family, friends, and colleagues. First, I would like to thank my advisor and committee chair, Dr. Natalia A. Schmid, for all of her support, advice, encouragement, and feedback along my research journey. The passion she showed me about her work and her knowledge from an engineering view point of radio astronomy inspired me to start researching. She's helped me understand the importance of research and has supported me throughout my undergraduate years and graduate school. She guided me on my undergraduate research for Poster's on the Hill in 2018, just as she's guided me throughout my graduate research. Without her guidance and direction I would not have found my love for research and would not have been able to complete this thesis. I would also like to thank Dr. Kevin Bandura for not only being a committee member but for also providing feedback and providing a clear direction for the next steps in this research. In addition, I would like to thank and extend my gratitude to Dr. Daryl Reynolds for being a committee member and providing me with feedback on how to improve my thesis. Next, I would like to thank Evan Smith and Pranav Sanghavi for sharing their radio astronomy data with me as well as answering any questions we had about this research. I would like to thank all of my friends and family. They were there for me whenever I needed extra support, encouragement, or a good laugh. My siblings and parents gave me their love, support, and encouragement from two-thousand miles and a phone call away. My in-law's offered me the same love, support and encouragement. My dog, Bart, was always there for me to love when I needed a puppy to cheer me up. Finally, I would like to extended my sincerest gratitude towards my loving husband, Jacob Dameron, who gave me the most encouragement, love, support, and feedback. He's been by my side every step of the way. He's helped me through every little hurdle and gave me a laugh or peace of mind whenever I needed it.
doi:10.33915/etd.11205 fatcat:6wohja5dzzaw7n3yjyhxoywh44