Performance Assertion Checking

Sharon E. Perl
1992 ACM SIGOPS Operating Systems Review  
Performance assertion checking is an approach to describing and monitoring the performance of complex software systems. The idea is simple: system implementors write assertions that capture their expectations for performance, the system is instrumented to collect performance data, and then the assertions are checked automatically against the data to detect violations signifying potential performance bugs. Because performance assertions provide a means of ltering data based on expectations, they
more » ... form a good basis for tools. Data indicating that a system is performing as expected can be discarded automatically, while data indicating potential problems can be brought to the attention of a person. Performance assertions are useful for performance regression testing, continuous system monitoring, and performance debugging. Also, the act of writing precise performance assertions helps designers and implementors to understand better their own expectations for performance, and the guarantees they can make about their systems. PSpec is a language and a set of tools that embodies the idea of performance assertion checking. A PSpec user writes performance assertions in the PSpec language. These are input, along with a monitoring log, to a checker tool that reports which assertions fail to hold for the log. Another tool called the solver helps to ll in constants in assertions using data in a monitoring log. PSpec has been used to nd performance bugs in the runtime system of a new parallel programming language, providing evidence of its utility. 4 During a fateful conversation about performance debugging tools and the lack thereof my advisor, Bill Weihl, asked, how about a tool that would check performance assertions?" It has taken almost four years to gure out what this question might mean and then to come up with an answer. Many people have helped along the way, and I am pleased to be able to acknowledge their contributions. My deepest appreciation and gratitude go to: Bill Weihl, who is everything a graduate student could want in an advisor, for his guidance, moral support and nancial support, friendship, and for maintaining his sanity during the past year while he was up for tenure, thereby helping me to maintain mine in my last year of graduate school. John Guttag, for being an involved and helpful memb e r o f m y thesis committee, for reminding me of the big picture when I got lost in the details, for having just the right w ords of wisdom when I needed them, for his eternal optimism and good cheer, and last but not least for lling a serious gap in my education concerning the history and virtues of the Larch tree. Roy Levin, whose participation on my thesis committee was above and beyond the call of duty, for making possible and enjoyable my three summers, one fall, one winter and one spring at DEC's Systems Research Center greatly stretching the notion of a summer intern", for acting as my advisor in residence at SRC, and for sharing his breadth of knowledge and good judgment about systems, from the lowest-level details to the highest-level concepts. Greg Nelson, for his unwavering and enthusiastic support of this research since I rst told him about it over a beer at Rudy's, for many signi cant technical contributions including the idea for the solver, for teaching me not to be afraid of simplicity, and for helping me nd the motivation both to keep going and to nish. Adrian Colbrook, for his energetic participation in the Prelude experiments that provided supporting evidence for the practicality of this research, and for much i n teresting gossip and philosophical discussion. 5 6
doi:10.1145/142111.964626 fatcat:36hb52xzz5e7rb2ifvg74vktgu