Pre-processing for Anomaly Detection on Linear Accelerator [report]

Martin Molan
2021 Zenodo  
This report describes the implementation of the data pre-processing for a novel anomaly detection technique. Proposed anomaly detection technique is based on using stochastic matrices as input for convolutional neural networks. Pre-processing step transforms raw data into 3d tensors combining stochastic matrices for a given event. Proposed solution for pre-processing is split into data reading part, which is parallelized with Dask and data processing part, which is parallelized with Spark. Data
more » ... ed with Spark. Data reading part runs on CERN's Swan notebook and data processing part runs on a Hadoop server (specifically general purpose server Analytics).
doi:10.5281/zenodo.4591034 fatcat:mtwfl6p5jzbc7drjdgha6amubi