A fully automatic method for recognizing hand configurations of Brazilian sign language

Cicero Ferreira Fernandes Costa Filho, Robson Silva de Souza, Jonilson Roque dos Santos, Bárbara Lobato dos Santos, Marly Guimarães Fernandes Costa
2017 Research on Biomedical Engineering  
Sign language is a collection of gestures, postures, movements, and facial expressions used by deaf people. The Brazilian sign language is Libras. The use of Libras has been increased among the deaf communities, but is still not disseminated outside this community. Sign language recognition is a field of research, which intends to help the deaf community communication with non-hearing-impaired people. In this context, this paper describes a new method for recognizing hand configurations of
more » ... figurations of Libras -using depth maps obtained with a Kinect  sensor. Methods: The proposed method comprises three phases: hand segmentation, feature extraction, and classification. The segmentation phase is independent from the background and depends only on pixel value. The feature extraction process is independent from rotation and translation. The features are extracted employing two techniques: (2D) 2 LDA and (2D) 2 PCA. The classification employs two classifiers: a novelty classifier and a KNN classifier. A robust database is constructed for classifier evaluation, with 12,200 images of Libras and 200 gestures of each hand configuration. Results: The best accuracy obtained was 96.31%. Conclusion: The best gesture recognition accuracy obtained is much higher than the studies previously published. It must be emphasized that this recognition rate is obtained for different conditions of hand rotation and proximity of the depth camera, and with a depth camera resolution of only 640×480 pixels. This performance must be also credited to the feature extraction technique, and to the size standardization and normalization processes used previously to feature extraction step. This is an Open Access article distributed under the terms of the Creative Commons Attribution License, which permits unrestricted use, distribution, and reproduction in any medium, provided the original work is properly cited. How to cite this article: Costa CFF Fo, Souza RS, Santos JR, Santos BL, Costa MGF. A fully automatic method for recognizing hand configurations of Brazilian sign language. Res Biomed Eng. 2017; 33(1):78-89.
doi:10.1590/2446-4740.03816 fatcat:sxrlbimdfjanrbc2w3wmz52er4