Automated mobile UI test fragility: an exploratory assessment study on Android

Riccardo Coppola, Emanuele Raffero, Marco Torchiano
<span title="">2016</span> <i title="ACM Press"> <a target="_blank" rel="noopener" href="" style="color: black;">Proceedings of the 2nd International Workshop on User Interface Test Automation - INTUITEST 2016</a> </i> &nbsp;
Automated UI testing suffers from fragility due to continuous -although minor -changes in the UI of applications. Such fragility has been shown especially for the web domain, though no clear evidence is available for mobile applications. Our goal is to perform an exploratory assessment of the extent and causes of the fragiliy of UI automated tests for mobile applications. For this purpose, we analyzed a small test suite -that we developed using five different testing frameworks-for an Android
more &raquo; ... plication (K-9 Mail) and observed the changes induced in the tests by the evolution of the UI. We found that up to 75% of code-based tests, and up to 100% of image recognition tests, had to be adapted because of the changes induced by the evolution of the application between two different versions. In addition we identified the main causes of such fragility: changes of identifiers, text or graphics, removal or relocation of elements, activity flow variation, execution time variation, and usage of physical buttons. The preliminary assessment showed that the fragility of UI tests can be a relevant issue also for mobile applications. A few common causes were found that can be used as the basis for providing guidelines for fragility avoidance and repair.
<span class="external-identifiers"> <a target="_blank" rel="external noopener noreferrer" href="">doi:10.1145/2945404.2945406</a> <a target="_blank" rel="external noopener" href="">dblp:conf/issta/CoppolaRT16</a> <a target="_blank" rel="external noopener" href="">fatcat:cfwnys6tofahtg5gsbqcypgdja</a> </span>
<a target="_blank" rel="noopener" href="" title="fulltext PDF download" data-goatcounter-click="serp-fulltext" data-goatcounter-title="serp-fulltext"> <button class="ui simple right pointing dropdown compact black labeled icon button serp-button"> <i class="icon ia-icon"></i> Web Archive [PDF] <div class="menu fulltext-thumbnail"> <img src="" alt="fulltext thumbnail" loading="lazy"> </div> </button> </a> <a target="_blank" rel="external noopener noreferrer" href=""> <button class="ui left aligned compact blue labeled icon button serp-button"> <i class="external alternate icon"></i> </button> </a>