Vision-based gesture recognition accords with natural communication habits of human and can carry out longdistance and non-contact interactions. So it has become a hot direction in human-computer interaction research whose recognition effect largely depends on the performance of image preprocessing and recognition algorithms. In this paper, a gesture recognition method using color image and depth image combined is designed. For the influence of the angle on the same gesture, the skeleton

more »

... hm is optimized based on the layerby-layer stripping concept. The fast refinement algorithm improves the process of repeated scanning, extracts the key node information in the skeleton map of the hand, and establishes the spatial axis of the hand to determine the gesture direction. The gesture recognition experiment was performed based on convolutional neural network. The results showed the recognition accuracy rate was 96.01%, and the robustness and accuracy of the proposed recognition method were verified.