Right for the Right Reasons: Training Differentiable Models by Constraining their Explanations

Andrew Slavin Ross, Michael C. Hughes, Finale Doshi-Velez
2017 Proceedings of the Twenty-Sixth International Joint Conference on Artificial Intelligence  
Expressive classifiers such as neural networks are among the most accurate supervised learning methods in use today, but their opaque decision boundaries make them difficult to trust in critical applications. We propose a method to explain the predictions of any differentiable model via the gradient of the class label with respect to the input (which provides a normal to the decision boundary). Not only is this approach orders of magnitude faster at identifying input dimensions of high
more » ... ty than sample-based perturbation methods (e.g. LIME), but it also lends itself to efficiently discovering multiple qualitatively different decision boundaries as well as decision boundaries that are consistent with expert annotation. On multiple datasets, we show our approach generalizes much better when test conditions differ from those in training.
doi:10.24963/ijcai.2017/371 dblp:conf/ijcai/RossHD17 fatcat:jqvkqmahtfftjk7b3janfdkglu