Safety Score as an Evaluation Metric for Machine Learning Models of Security Applications

Tara Salman, Ali Ghubaish, Devrim Unal, Raj Jain
2020 IEEE Networking Letters  
Machine learning studies have traditionally used accuracy, F 1 score, etc. to measure the goodness of models. We show that these conventional metrics do not necessarily represent risks in security applications and may result in models that are not optimal. This letter proposes 'Safety score' as an evaluation metric that incorporates the cost associated with model predictions. The proposed metric is easy to explain to system administrators. We evaluate the new metric for two security
more » ... : general intrusion detection and injection attack detection. Compared to other metrics, Safety score proves its efficiency in indicating the risk in using the model.
doi:10.1109/lnet.2020.3016583 fatcat:s3igiiwwordrrcqktrafc64zsq