Development of Learning Support System for Fingerspelling by Augmented Reality

Yoshinori Fujisawa, Shoichi Ito, Kenya Kobayashi
2017 Proceedings of The 5th IIAE International Conference on Industrial Application Engineering 2017   unpublished
Our purpose of this study is to develop a learning support system for hearing-impaired people who learn sign language and fingerspelling alone. Fingerspelling is one type of the visual language which is used among hearingimpaired people as a communication tool. One fingerspell letter is represented by the shape of communicator's hands, and it corresponds to one letter of a writing system. Because of the simple structure of fingerspelling, it is used for proper nouns and some words which are
more » ... words which are difficult to express as a native word of sign language. In general, learners of fingerspelling refer to photographs or illustrations. These learning materials are described from the opposite point of view of learners in most cases. Therefore, the best way to learn fingerspelling is to take lessons conducted by a teacher who has already mastered it. It is desirable for learners to learn anytime they want. However, it is difficult to find the best schedules for both teacher and learners. We propose a new learning support system using augmented reality. Our system is implemented using smart glasses. In this system, finger shapes are displayed on the internal display of the smart glasses so that user can be given a sense that the user is looking at the fingerspelling shape in the real world, and thus, the user can grasp the finger shapes intuitively. In this paper, we show how the system works, and evaluate the efficiency of learning fingerspelling by using this system.
doi:10.12792/icisip2017.090 fatcat:k33ibnhgtrfetabcionqwv7nqy