Non-native speaker perception of Intelligent Virtual Agents in two languages: the impact of amount and type of grammatical mistakes

David Obremski, Jean-Luc Lugrin, Philipp Schaper, Birgit Lugrin
2021 Journal on Multimodal User Interfaces  
AbstractHaving a mixed-cultural membership becomes increasingly common in our modern society. It is thus beneficial in several ways to create Intelligent Virtual Agents (IVAs) that reflect a mixed-cultural background as well, e.g., for educational settings. For research with such IVAs, it is essential that they are classified as non-native by members of a target culture. In this paper, we focus on variations of IVAs' speech to create the impression of non-native speakers that are identified as
more » ... uch by speakers of two different mother tongues. In particular, we investigate grammatical mistakes and identify thresholds beyond which the agents is clearly categorised as a non-native speaker. Therefore, we conducted two experiments: one for native speakers of German, and one for native speakers of English. Results of the German study indicate that beyond 10% of word order mistakes and 25% of infinitive mistakes German-speaking IVAs are perceived as non-native speakers. Results of the English study indicate that beyond 50% of omission mistakes and 50% of infinitive mistakes English-speaking IVAs are perceived as non-native speakers. We believe these thresholds constitute helpful guidelines for computational approaches of non-native speaker generation, simplifying research with IVAs in mixed-cultural settings.
doi:10.1007/s12193-021-00369-9 fatcat:nmidcnqrvbdp5m4meyfnuuzpxi