Evaluating American Sign Language generation through the participation of native ASL signers

Matt Huenerfauth, Liming Zhao, Erdan Gu, Jan Allbeck
2007 Proceedings of the 9th international ACM SIGACCESS conference on Computers and accessibility - Assets '07  
We discuss important factors in the design of evaluation studies for systems that generate animations of American Sign Language (ASL) sentences. In particular, we outline how some cultural and linguistic characteristics of members of the American Deaf community must be taken into account so as to ensure the accuracy of evaluations involving these users. Finally, we describe our implementation and user-based evaluation (by native ASL signers) of a prototype ASL generator to produce sentences
more » ... oduce sentences containing classifier predicates, frequent and complex spatial phenomena that previous ASL generators have not produced. Since there have been few sign language generation or machine translation systems developed, there has been correspondingly little work on how to best evaluate such systems. Broadly speaking, evaluations of natural language generation software fall into two major categories: automated and user-based. We will discuss how the special linguistic properties of American Sign
doi:10.1145/1296843.1296879 dblp:conf/assets/HuenerfauthZGA07 fatcat:23h6jgv3kbfvxj34fbek42leea