LINEAR: A MULTI-DEVICE AUGMENTED REALITY ENVIRONMENT FOR INTERACTIVE NOTATION AND MUSIC IMPROVISATION

Giovanni Santini, Tiffon Vincent, Bell Jonathan, de Paiva Santana Charles
2022 Zenodo  
LINEAR (Live-generated Interface and Notation Environment in Augmented Reality) is an environment for the generation of real-time 3D interactive graphic notation. The environment is suitable for ensemble improvisative performances featuring acoustic instruments, live-electronics and two Augmented Reality (AR) performers. One AR performer uses an iPhone for drawing virtual trajectories in the space, rendered as a sequence of Virtual Objects (VOs) aligned along the trajectory. VOs trigger samples
more » ... upon virtual collisions with the iPhone. They are also used as a form of graphic notation for instrumentalists/vocalists: the screen of the iPhone is mirrored to a projector. The second AR performer uses a headset and can use VR controllers to design trajectories used for the spatialization of each audio source in a 3D audio setup. The headset AR performer can use virtual spheres (one per instrument) to control the posi- tion of each sound source (one per instrument). The sound of every acoustic instrument is processed live. The mixing of processing effects are controlled by a laptop player. The system has been repeatedly tested during a two-semesters long workshop. The system was also used for two online concerts. Beyond demonstrating the technical and musical viability of LINEAR, the workshop also gave the chance to record student's response to the system. Although the sample size is quite small (four students completed the survey), the answers show encouraging results in terms of engagement and interest. Future work should be conducted to further enhance the user experience and more clearly assess LINEAR's usability and effectiveness as an innovative system for improvisation and musical performance.
doi:10.5281/zenodo.6669016 fatcat:mbtjlztzejbc7hpe4vm5n56rku