Exploiting Verified Neural Networks via Floating Point Numerical Error [article]

Kai Jia, Martin Rinard
<span title="2021-10-01">2021</span> <i > arXiv </i> &nbsp; <span class="release-stage" >pre-print</span>
Researchers have developed neural network verification algorithms motivated by the need to characterize the robustness of deep neural networks. The verifiers aspire to answer whether a neural network guarantees certain properties with respect to all inputs in a space. However, many verifiers inaccurately model floating point arithmetic but do not thoroughly discuss the consequences. We show that the negligence of floating point error leads to unsound verification that can be systematically
more &raquo; ... ited in practice. For a pretrained neural network, we present a method that efficiently searches inputs as witnesses for the incorrectness of robustness claims made by a complete verifier. We also present a method to construct neural network architectures and weights that induce wrong results of an incomplete verifier. Our results highlight that, to achieve practically reliable verification of neural networks, any verification system must accurately (or conservatively) model the effects of any floating point computations in the network inference or verification system.
<span class="external-identifiers"> <a target="_blank" rel="external noopener" href="https://arxiv.org/abs/2003.03021v4">arXiv:2003.03021v4</a> <a target="_blank" rel="external noopener" href="https://fatcat.wiki/release/6c3gysniajdxpb4wmzumoecj7m">fatcat:6c3gysniajdxpb4wmzumoecj7m</a> </span>
<a target="_blank" rel="noopener" href="https://web.archive.org/web/20211006093725/https://arxiv.org/pdf/2003.03021v4.pdf" title="fulltext PDF download" data-goatcounter-click="serp-fulltext" data-goatcounter-title="serp-fulltext"> <button class="ui simple right pointing dropdown compact black labeled icon button serp-button"> <i class="icon ia-icon"></i> Web Archive [PDF] <div class="menu fulltext-thumbnail"> <img src="https://blobs.fatcat.wiki/thumbnail/pdf/c5/91/c591c18da00a8e7fb4efda480917ef1d007f2e15.180px.jpg" alt="fulltext thumbnail" loading="lazy"> </div> </button> </a> <a target="_blank" rel="external noopener" href="https://arxiv.org/abs/2003.03021v4" title="arxiv.org access"> <button class="ui compact blue labeled icon button serp-button"> <i class="file alternate outline icon"></i> arxiv.org </button> </a>