Exploratory Datamorphic Testing of Classification Applications

Hong Zhu, Ian Bayley
2020 Proceedings of the IEEE/ACM 1st International Conference on Automation of Software Test  
Testing has been widely recognised as difficult for AI applications. This paper proposes a set of testing strategies for testing machine learning applications in the framework of the datamorphism testing methodology. In these strategies, testing aims at exploring the data space of a classification or clustering application to discover the boundaries between classes that the machine learning application defines. This enables the tester to understand precisely the behaviour and function of the
more » ... tware under test. In the paper, three variants of exploratory strategies are presented with the algorithms as implemented in the automated datamorphic testing tool Morphy. The correctness of these algorithms are formally proved. The paper also reports the results of some controlled experiments with Morphy that study the factors that affect the test effectiveness of the strategies. CCS CONCEPTS • Software and its engineering → Software notations and tools; • Computing methodologies → Artificial intelligence; Machine learning.
doi:10.1145/3387903.3389312 dblp:conf/icse/0002B20 fatcat:rbmtwwle6jcl5eqd66r3um2qv4