Speaking with hands

Matthew Stone, Doug DeCarlo, Insuk Oh, Christian Rodriguez, Adrian Stere, Alyssa Lees, Chris Bregler
2004 ACM Transactions on Graphics  
Figure 1 : "Ow, dude! You missed your cue to take hold of the board!" Clips of sound and clips of motion are selected from separate performances, spliced together and resynchronized to create the animated delivery of a meaningful new utterance. Abstract We describe a method for using a database of recorded speech and captured motion to create an animated conversational character. People's utterances are composed of short, clearly-delimited phrases; in each phrase, gesture and speech go together
more » ... meaningfully and synchronize at a common point of maximum emphasis. We develop tools for collecting and managing performance data that exploit this structure. The tools help create scripts for performers, help annotate and segment performance data, and structure specific messages for characters to use within application contexts. Our animations then reproduce this structure. They recombine motion samples with new speech samples to recreate coherent phrases, and blend segments of speech and motion together phraseby-phrase into extended utterances. By framing problems for utterance generation and synthesis so that they can draw closely on a talented performance, our techniques support the rapid construction of animated characters with rich and appropriate expression.
doi:10.1145/1015706.1015753 fatcat:lbvcjr4wkraatmx74qr3frvmaq