An Analysis of Case-Base Editing in a Spam Filtering System [chapter]

Sarah Jane Delany, Pádraig Cunningham
2004 Lecture Notes in Computer Science  
Because of the volume of spam email and its evolving nature, any deployed Machine Learning-based spam filtering system will need to have procedures for case-base maintenance. Key to this will be procedures to edit the case-base to remove noise and eliminate redundancy. In this paper we present a two stage process to do this. We present a new noise reduction algorithm called Blame-Based Noise Reduction that removes cases that are observed to cause misclassification. We also present an algorithm
more » ... alled Conservative Redundancy Reduction that is much less aggressive than the state-of-the-art alternatives and has significantly better generalisation performance in this domain. These new techniques are evaluated against the alternatives in the literature on four datasets of 1000 emails each (50% spam and 50% non spam).
doi:10.1007/978-3-540-28631-8_11 fatcat:o2iy3qrowrbldfh3m6ipigcbca