A Scalable and Efficient Outlier Detection Strategy for Categorical Data

Anna Koufakou, Enrique G. Ortiz, Michael Georgiopoulos, Georgios C. Anagnostopoulos, Kenneth M. Reynolds
2007 19th IEEE International Conference on Tools with Artificial Intelligence(ICTAI 2007)  
Outlier detection has received significant attention in many applications, such as detecting credit card fraud or network intrusions. Most existing research focuses on numerical datasets, and cannot directly apply to categorical sets where there is little sense in calculating distances among data points. Furthermore, a number of outlier detection methods require quadratic time with respect to the dataset size and usually multiple dataset scans. These characteristics are undesirable for large
more » ... asets, potentially scattered over multiple distributed sites. In this paper, we introduce Attribute Value Frequency (AVF), a fast and scalable outlier detection strategy for categorical data. AVF scales linearly with the number of data points and attributes, and relies on a single data scan. AVF is compared with a list of representative outlier detection approaches that have not been contrasted against each other. Our proposed solution is experimentally shown to be significantly faster, and as effective in discovering outliers.
doi:10.1109/ictai.2007.125 dblp:conf/ictai/KoufakouOGAR07 fatcat:cauqplj4zvdtrar2pa6anwji5y