Large-Vocabulary Continuous Sign Language Recognition Based on Transition-Movement Models

Gaolin Fang, Wen Gao, Debin Zhao
2007 IEEE transactions on systems, man and cybernetics. Part A. Systems and humans  
The major challenges that sign language recognition (SLR) now faces are developing methods that solve largevocabulary continuous sign problems. In this paper, transitionmovement models (TMMs) are proposed to handle transition parts between two adjacent signs in large-vocabulary continuous SLR. For tackling mass transition movements arisen from a large vocabulary size, a temporal clustering algorithm improved from k-means by using dynamic time warping as its distance measure is proposed to
more » ... s proposed to dynamically cluster them; then, an iterative segmentation algorithm for automatically segmenting transition parts from continuous sentences and training these TMMs through a bootstrap process is presented. The clustered TMMs due to their excellent generalization are very suitable for large-vocabulary continuous SLR. Lastly, TMMs together with sign models are viewed as candidates of the Viterbi search algorithm for recognizing continuous sign language. Experiments demonstrate that continuous SLR based on TMMs has good performance over a large vocabulary of 5113 Chinese signs and obtains an average accuracy of 91.9%. Index Terms-Chinese sign language (CSL), dynamic time warping (DTW), hidden Markov model (HMM), sign language recognition (SLR), temporal clustering algorithm.
doi:10.1109/tsmca.2006.886347 fatcat:2riou3fzonetdlnigj6asvv6o4