A Multimodal System for Public Speaking with Real Time Feedback

Fiona Dermody, Alistair Sutherland
2015 Proceedings of the 2015 ACM on International Conference on Multimodal Interaction - ICMI '15  
We have developed a multimodal prototype for public speaking with real time feedback using the Microsoft Kinect. Effective speaking involves use of gesture, facial expression, posture, voice as well as the spoken word. These modalities combine to give the appearance of self-confidence in the speaker. This initial prototype detects body pose, facial expressions and voice. Visual and text feedback is displayed in real time to the user using a video panel, icon panel and text feedback panel. The
more » ... edback panel. The user can also set and view elapsed time during their speaking performance. Real time feedback is displayed on gaze direction, body pose and gesture, vocal tonality, vocal dysfluencies and speaking rate.
doi:10.1145/2818346.2823295 dblp:conf/icmi/DermodyS15 fatcat:hnbyoe3ht5gs3jl7ysqvalqxre