3D natural hand interaction for AR applications

M. Lee, R. Green, M. Billinghurst
2008 2008 23rd International Conference Image and Vision Computing New Zealand  
In this paper we describe a computer vision-based 3D hand tracking system for a multimodal Augmented Reality (AR) interface. We have developed a 3D vision-based natural hand interaction method. This consists of four steps: (1) Skin colour segmentation, (2) Feature point finding, (3) Hand direction calculation, and (4) Simple collision detection based on a short finger ray for interaction between the user's hand and augmented objects. The resulting fingertip tracking accuracy varied from 3mm to
more » ... 0mm depending on the distance between the user's hand and the stereo camera. We describe how this hand tracking is applied in three AR applications which merge gesture and speech input.
doi:10.1109/ivcnz.2008.4762125 fatcat:cgk7frvq2jhulh6pzjgqpff3ya