The Literature Survey on Virtual Piano

Yashwanth G, Saifulla Khan, T V Rahul Reddy, Sukanya H A, Varsha N, Vidya Vikas Institute of Engineering Technology
2020 International Journal of Engineering Research and  
This paper presents an efficient data-driven approach to track fingertip and detect finger tapping for virtual piano using an RGB-D camera. We collect 7200 depth images covering the most common finger articulation for playing piano, and train a random regression forest using depth context features of randomly sampled pixels in training images. In the online tracking stage, we firstly segment the hand from the plane in contact by fusing the information from both color and depth images. Then we
more » ... e the trained random forest to estimate the 3D position of fingertips and wrist in each frame, and predict finger tapping based on the estimated fingertip motion. Finally, we build a kinematic chain and recover the articulation parameters for each finger. In contrast to the existing hand tracking algorithms that often require hands are in the air and cannot interact with physical objects, our method is designed for hand interaction with planar objects, which is desired for the virtual piano application. Using our prototype system, users can put their hands on a desk, move them sideways and then tap fingers on the desk, like playing a real piano. Preliminary results show that our method can recognize most of the beginner's piano-playing gestures in real-time for soothing rhythms.
doi:10.17577/ijertv9is040405 fatcat:cohbnqipr5ftdjulci2nkgqspa