Why Can't Johnny Fix Vulnerabilities: A Usability Evaluation of Static Analysis Tools for Security

Justin Smith, Lisa Nguyen Quang Do, Emerson R. Murphy-Hill
2020 Symposium On Usable Privacy and Security  
Static analysis tools can help prevent security incidents, but to do so, they must enable developers to resolve the defects they detect. Unfortunately, developers often struggle to interact with the interfaces of these tools, leading to tool abandonment, and consequently the proliferation of preventable vulnerabilities. Simply put, the usability of static analysis tools is crucial. The usable security community has successfully identified and remedied usability issues in end user security
more » ... ations, like PGP and Tor browsers, by conducting usability evaluations. Inspired by the success of these studies, we conducted a heuristic walkthrough evaluation and user study focused on four security-oriented static analysis tools. Through the lens of these evaluations, we identify several issues that detract from the usability of static analysis tools. The issues we identified range from workflows that do not support developers to interface features that do not scale. We make these findings actionable by outlining how our results can be used to improve the state-of-the-art in static analysis tool interfaces. Introduction Security-oriented static analysis tools, like Spotbugs [12], Checkmarx [2], and CodeSonar [3] enable developers to detect issues early in the development process. Among several types of code quality issues, developers rank security issues as the highest priority for these tools to detect [22] . Evaluating the efficacy of these security-oriented static analysis tools has been a popular topic for researchers [17, 29,
dblp:conf/soups/0001DM20 fatcat:ddt6hgrb2bhihm6fugmzlw6mda