While a lot of progress has been made in improving analyses and tools that aid software development, little effort has been spent on studying how such tools are commonly used in practice. A study into a tool's usage is important not only because it can help improve the tool's usability but also because it can provide key insights for improving the tool's underlying analysis technology in a common usage scenario. This paper presents a study that explores how (beginner) users work with the Alloy

more »

... nalyzer, a tool for automatic analysis of software models written in Alloy, a first-order, declarative language. Alloy has been successfully used in research and teaching for several years, but there has been no study of how users interact with the analyzer. We have modified the analyzer to log (some of) its interactions with the user. Using this modified analyzer, students in two of our (graduate) classes formulated their Alloy models to solve a problem set. Our analysis of the resulting logs shows several interesting observations, and based on them, we propose how to improve the analyzer (both the performance of analyses and the user interaction). Specifically, we show that: (i) users often perform consecutive analyses with slightly different models, and thus incremental analysis can speed up the interaction; (ii) users' interaction with the analyzer are sometimes predictable, and akin to continuous compilation, the analyzer can precompute the result of a future action while the user is editing the model; and (iii) (beginner) users can naturally develop semantically equivalent models that have vastly different analysis time, which refutes a previous belief that semantically equivalent Alloy models tend to have similar analysis time.