The Mouthesizer: A Facial Gesture Musical Interface

Michael Lyons, Michael Haehnel, Nobuji Tetsutani
2017 Figshare  
In this technical sketch we introduce a new interaction paradigm: using facial gestures and expressions to control musical sound. The mouthesizer, a prototype of this general concept, consists of a miniature head mounted camera which acquires video input from the region of the mouth, extracts the mouth shape using a simple computer vision algorithm, and converts shape parameters to MIDI commands to control a synthesizer or musical effects device.
doi:10.6084/m9.figshare.5248165 fatcat:62wx4bpjzna4fjyvrbbujlwngy