User-Centric Context-Aware Mobile Applications for Embodied Music Listening
Lecture Notes of the Institute for Computer Sciences, Social Informatics and Telecommunications Engineering
This paper surveys a collection of sample applications for networked user-centric context-aware embodied music listening. The applications have been designed and developed in the framework of the EU-ICT Project SAME (www.sameproject.eu) and have been presented at Agora Festival (IRCAM, Paris, France) in June 2009. All of them address in different ways the concept of embodied, active listening to music, i.e., enabling listeners to interactively operate in real-time on the music content by means
... f their movements and gestures as captured by mobile devices. In the occasion of the Agora Festival the applications have also been evaluated by both expert and non-expert users. 2 A. Camurri, G. Volpe, H. Vinet, R. Bresin, M. Fabiani, G. Dubus, E. Maestre, J. Llop, J. Kleimola, S. Oksanen, V. Välimäki, J. Seppanen environments. The project is based on the concept of active listening, i.e., listeners are enabled to interactively operate on (pre-recorded) music content through their movement and gesture, by modifying and molding it in real-time while listening. This is obtained through the development of a networked end-to-end platform for mobile music applications enabling novel paradigms for natural, expressive/emotional multimodal interfaces, empowering the user to influence, interact, mould and shape the music content, by intervening actively and physically into the experience. Active listening is the basic concept for a novel generation of interactive music applications, particularly addressed to a general public of beginners, naïve and inexperienced users, rather than to professional musicians. A particularly relevant aspect of active listening is its social, collaborative implication: active listening enables a social, collaborative, and context aware experience of music, allowing listeners to cooperate in the real-time manipulation and re-creation of music content. Examples of the active listening paradigm are emerging. The Orchestra Explorer  enables users to explore a space populated by virtual instruments. Mappe per Affetti Erranti  introduces multiple levels of navigation: from navigation in a physical space up to emotional spaces populated by different expressive performances of the same music piece. Users can navigate such spaces by their expressive movement and gesture. Mappe per Affetti Erranti also addresses experience by multiple users encouraging social behavior. The virtual air guitars [3, 4] are examples of gesture-based mobile musical instruments. They can be made easier to play than conventional musical instruments, because user's gestures can be interpreted by the computer to produce the desired output sound. This paper surveys a first set of such mobile context-aware music applications, presented by the SAME partners at the Agora Festival (IRCAM, Paris, France, June 2009) and representing the mid-term milestone of the project (Section 2). Prototype applications were evaluated by both expert and non-expert users visiting the festival. Results from such evaluation are also discussed (Section 3).