A Controlled Experiment for Program Comprehension through Trace Visualization

Bas Cornelissen, Andy Zaidman, Arie van Deursen
2011 IEEE Transactions on Software Engineering  
Software maintenance activities require a sufficient level of understanding of the software at hand that unfortunately is not always readily available. Execution trace visualization is a common approach in gaining this understanding, and among our own efforts in this context is EXTRAVIS, a tool for the visualization of large traces. While many such tools have been evaluated through case studies, there have been no quantitative evaluations to the present day. This paper reports on the first
more » ... olled experiment to quantitatively measure the added value of trace visualization for program comprehension. We designed eight typical tasks aimed at gaining an understanding of a representative subject system, and measured how a control group (using the Eclipse IDE) and an experimental group (using both Eclipse and EXTRAVIS) performed these tasks in terms of time spent and solution correctness. The results are statistically significant in both regards, showing a 22% decrease in time requirements and a 43% increase in correctness for the group using trace visualization. Index Terms-Program comprehension, dynamic analysis, controlled experiment. ! • B. Cornelissen is with the Software Improvement Group, A.J. Ernststraat
doi:10.1109/tse.2010.47 fatcat:rrcso2vzg5c2tn7gkqrzb4fvou