Smartwatch-based Early Gesture Detection 8 Trajectory Tracking for Interactive Gesture-Driven Applications

Tran Huy Vu, Archan Misra, Quentin Roy, Kenny Choo Tsu Wei, Youngki Lee
2018 Proceedings of the ACM on Interactive Mobile Wearable and Ubiquitous Technologies  
Smartwatch-based early gesture detection & trajectory tracking for interactive gesture-driven applications. (2018). The paper explores the possibility of using wrist-worn devices (specifically, a smartwatch) to accurately track the hand movement and gestures for a new class of immersive, interactive gesture-driven applications. These interactive applications need two special features: (a) the ability to identify gestures from a continuous stream of sensor data early-i.e., even before the
more » ... is complete, and (b) the ability to precisely track the hand's trajectory, even though the underlying inertial sensor data is noisy. We develop a new approach that tackles these requirements by first building a HMM-based gesture recognition framework that does not need an explicit segmentation step, and then using a per-gesture trajectory tracking solution that tracks the hand movement only during these predefined gestures. Using an elaborate setup that allows us to realistically study the table-tennis related hand movements of users, we show that our approach works: (a) it can achieve 95% stroke recognition accuracy. Within 50% of gesture, it can achieve a recall value of 92% for 10 novice users and 93% for 15 experienced users from a continuous sensor stream; (b) it can track hand movement during such strokeplay with a median accuracy of 6.2 cm.
doi:10.1145/3191771 fatcat:ypc7cm455rhmtakx3xtqku7x44