Manifold Regularization for Locally Stable Deep Neural Networks [article]

Charles Jin, Martin Rinard
2020 arXiv   pre-print
We apply concepts from manifold regularization to develop new regularization techniques for training locally stable deep neural networks. Our regularizers are based on a sparsification of the graph Laplacian which holds with high probability when the data is sparse in high dimensions, as is common in deep learning. Empirically, our networks exhibit stability in a diverse set of perturbation models, including ℓ_2, ℓ_∞, and Wasserstein-based perturbations; in particular, we achieve 40 against an
more » ... daptive PGD attack using ℓ_∞ perturbations of size ϵ = 8/255, and state-of-the-art verified accuracy of 21 perturbation model. Furthermore, our techniques are efficient, incurring overhead on par with two additional parallel forward passes through the network.
arXiv:2003.04286v2 fatcat:7v5tuul45vgy7jiljtannwg6um