Speech, ink, and slides

Richard Anderson, Crystal Hoyer, Craig Prince, Jonathan Su, Fred Videon, Steve Wolfman
2004 Proceedings of the 12th annual ACM international conference on Multimedia - MULTIMEDIA '04  
In this paper, we report on an empirical exploration of digital ink and speech usage in lecture presentation. We studied the video archives of five Master's level Computer Science courses to understand how instructors use ink and speech together while lecturing, and to evaluate techniques for analyzing digital ink. Our interest in understanding how ink and speech are used together is to inform the development of future tools for supporting classroom presentation, distance education, and viewing
more » ... of archived lectures. We want to make it easier to interact with electronic materials and to extract information from them. We want to provide an empirical basis for addressing challenging problems such as automatically generating full text transcripts of lectures, matching speaker audio with slide content, and recognizing the meaning of the instructor's ink. Our results include an evaluation of handwritten word recognition in the lecture domain, an approach for associating attentional marks with content, an analysis of linkage between speech and ink, and an application of recognition techniques to infer speaker actions.
doi:10.1145/1027527.1027713 dblp:conf/mm/AndersonHPSVW04 fatcat:nmkzm26c55dnhhy423tqmxctdu