Anywhere touchtyping

Adiyan Mujibiya, Takashi Miyaki, Jun Rekimoto
2010 Adjunct proceedings of the 23nd annual ACM symposium on User interface software and technology - UIST '10  
Camera Fringe pattern Text input Figure 1: The actual system and its components of our real-time hand posture depth sensing for touch typing on arbitrary surface. (a) Hardware configuration: DLP projector dispatch fringe pattern onto the surface and video camera captures the projection area. (b) Computer vision software result for depth sensing, virtual keyboard area estimation, fingertip detection, and keystroke event sensing. (c) Usage scenario: users put their hands in the depth sensible
more » ... of the system and perform touch typing. ABSTRACT In this paper, touch typing enabled virtual keyboard system using depth sensing on arbitrary surface is proposed. Keystroke event detection is conducted using 3-dimensional hand appearance database matching combined with fingertip's surface touch sensing. Our prototype system acquired hand posture depth map by implementing phase shift algorithm for Digital Light Processor (DLP) fringe projection on arbitrary flat surface. The system robustly detects hand postures on the sensible surface with no requirement of hand position alignment on virtual keyboard frame. The keystroke feedback is the physical touch to the surface, thus no specific hardware must be worn. The system works real-time in average of 20 frames per second.
doi:10.1145/1866218.1866262 dblp:conf/uist/MujibiyaMR10 fatcat:uzbp5j4ym5asvkonsycxyb3oyy