Speech-gesture driven multimodal interfaces for crisis management

R. Sharma, M. Yeasin, N. Krahntoever, I. Rauschert, Guoray Cai, I. Brewer, A.M. Maceachren, K. Sengupta
2003 Proceedings of the IEEE  
Emergency response requires strategic assessment of risks, decisions, and communications that are timecritical while requiring teams of individuals to have fast access to large volumes of complex information and technologies that enables tightly coordinated work. The access to this information by crisis management (CM) teams in emergency operations centers can be facilitated through various humancomputer interfaces. Unfortunately these interfaces are hard to use, require extensive training, and
more » ... often impede rather than support teamwork. Dialogue-enabled devices, based on natural, multimodal interfaces have the potential of making a variety of information technology tools accessible during crisis management. This paper establishes the importance of multimodal interfaces in various aspects of crisis management and explores many issues in realizing successful speech-gesture driven, dialog-enabled interfaces for CM. The paper is organized in five parts. The first part discusses the needs of CM that can be potentially met by the development of appropriate interfaces. The second part discusses the issues related to the design and development of multimodal interfaces in the context of CM. The third part discusses the state of the art in both the theories and practices involving these human-computer interfaces. In particular it describes, the evolution and implementation details of two representative systems, called crisis management (XISM) and Dialog Assisted Visual Environment for Geoinformation (DAVE_G). The fourth part speculates on the short term and long term research directions that will help addressing the outstanding challenges in interfaces that support dialog and collaboration. Finally, part five concludes the paper.
doi:10.1109/jproc.2003.817145 fatcat:flbaisvreresla7wufztzpnvfq