Uncovering Unknown Unknowns in Financial Services Big Data by Unsupervised Methodologies: Present and Future trends

Gil Shabat, David Segev, Amir Averbuch
2017 Knowledge Discovery and Data Mining  
Currently, unknown unknowns in high dimensional big data environments can go unnoticed for a long period of time. The failure to detect anomalies in critical infrastructure data can result in extensive financial, operational, reputational and life threatening consequences. In this paper, we describe algorithms for an automatic and unsupervised anomaly detection that do not necessitate domain expertise, signatures, rules, patterns or semantics understanding of the features. We propose several
more » ... methodologies for anomaly detection to protect critical infrastructures, with emphasis on finance, spanning from theory to actionable technology. Although anomalies can originate from several sources, we also show that cyber threat, financial and operational malfunction are converging into a single detection paradigm. Performance comparison between different algorithms (ours and others) is presented as well as examples from real use cases.
dblp:conf/kdd/ShabatSA17 fatcat:2w6atpdtrndtxf6qo37ddcrstm