Emergent Perception and Video Games that Listen: Applying Sonic Virtuality for Creative and Intelligent Virtual Agent Behaviours

Anna Jordanous, Tom Garner
2016 Computational Creativity & Games Workshop (CCGW)  
Computer-controlled virtual agents, typically referred to in the video games industry as 'nonplayer characters (NPCs)' can present well-crafted behaviours and evoke engaging and immersive player-experiences. One particular class of behaviours relates to those that portray creative problem solving; the generation of ideas, emerging from the situation and their 'character', to overcome problems and advance the narrative. However, such behaviour in current NPCs is arguably illusory and is only
more » ... evable within a controlled and linear video game context. NPCs struggle greatly to portray flexible or creative behaviours within an adaptive or procedurally generated environment and this is even more apparent in their relationship with sound. Contemporary psychology and neuroscience research increasingly advocates embodied cognition perspectives and this paper posits that such theoretical developments offer significant opportunity to advance NPC-AI. This paper proposes that an intelligence framework based upon the recently published theory of Sonic Virtuality, and integrated within an NPC, would offer distinct advantages over current systems. With regards to computational creativity, the particular progression for NPCs relates to creative problem solving. To illustrate this vision, a roadmap for future work is laid out, using Sonic Virtuality as the foundation for a 'synthetic listener', an NPC capable of responding to procedurally generated and external (playerdomain) audio. The closing section of this paper further posits that the principles of Sonic Virtuality could be applied equally well to other perception modalities such as visual feedback, presenting a unique avenue of AI research that, ultimately, could dramatically improve NPC's 'humanness' and enable them to evoke a player's sense of immersion and presence that is equivalent to existing scripted AI but in much bigger, more complex virtual worlds.