Latent Mappings: Generating Open-Ended Expressive Mappings Using Variational Autoencoders [article]

Tim Murray-Browne, Panagiotis Tigas
2021 arXiv   pre-print
In many contexts, creating mappings for gestural interactions can form part of an artistic process. Creators seeking a mapping that is expressive, novel, and affords them a sense of authorship may not know how to program it up in a signal processing patch. Tools like Wekinator and MIMIC allow creators to use supervised machine learning to learn mappings from example input/output pairings. However, a creator may know a good mapping when they encounter it yet start with little sense of what the
more » ... puts or outputs should be. We call this an open-ended mapping process. Addressing this need, we introduce the latent mapping, which leverages the latent space of an unsupervised machine learning algorithm such as a Variational Autoencoder trained on a corpus of unlabelled gestural data from the creator. We illustrate it with Sonified Body, a system mapping full-body movement to sound which we explore in a residency with three dancers.
arXiv:2106.08867v1 fatcat:axmw3w2oljhl7lyopp6jqqot2q