False Positives (FPs) and False Negatives (FNs) happen to every Intrusion Detection/Prevention System (IDS/IPS). This work proposes a mechanism of False Positive/Negative Assessment (FPNA) with multiple IDSs/IPSs to collect FP and FN cases from real-world traffic. Over a period of sixteen months, more than two thousand FPs and FNs have been collected and analyzed. From the statistical analysis results, we obtain three interesting findings. First, more than 92.85% of false cases are FPs even if

more »

... he numbers of attack types for FP and FN are similar. Second, about 91% of FP alerts, equal to about 85% of false cases, are not related to security issues, but to management policy. The last finding shows that buffer overflow, SQL server attack and worm slammer attacks account for 93% of FNs, even though they are aged attacks. This indicates that these attacks always have new variations to evade IDS/IPS detection.