Virtual and Mixed Reality in Telerobotics: A Survey [chapter]

Costas S.
2006 Industrial Robotics: Programming, Simulation and Applications  
of interacting within this world. The auxiliary information that can be conveyed through the visual, haptic, or other form of display of virtual models may facilitate the user to perceive in a more direct and intuitive way specific "hidden" or else "fuzzy" characteristics of a real world, which may be needed for the efficient execution of a certain task. Therefore, while in general VR promises to revolutionize the way human-computer interaction systems are conceived, AR techniques seem to lead
more » ... o the creation of excellent tools making execution of complex and demanding tasks more easy and intuitive for the human. Virtual, Augmented and Mixed Reality technologies are now recognized as constituting a challenging scientific and technological field that can provide breakthrough solutions to a wide spectrum of application domains, where intuitive and natural human/computer and human/machine interaction is needed. Telerobotics, involving a human operator to control a robot from a remote location usually via a computer interface and a computer network, is one of the fields that can directly benefit from the potential offered by VR and AR human/machine interfacing technologies. Application of VR in Telerobotics (VRT) and the related concept of telepresence, or tele-symbiosis (Vertut & Coiffet, 1984) , are ideas which have been a r o u n d f o r m o r e t h a n t w e n t y y e a r s , a n d h a v e b e e n u s e d m a i n l y f o r t h e telemanipulation/teleoperation of robotic mechanisms in hostile environments (such as in the nuclear industry, space, underwater, or for other difficult or dangerous service/intervention tasks, like bomb disposal, civil works etc.). In fact, all the approaches involving the integration of VR techniques in telerobotics, as will be explained further in this chapter, constitute basically: (a) a generalization of the concept of "predictive displays", coping with the problem of time delay and stability in teleoperation systems, and (b) an attempt to provide human operator assistance and achieve better transparency characteristics for the teleoperation system. Nowadays, on the other hand, the rapid development of new broadly expanded networking technologies, such as those related to the Internet, and the numerous relevant applications, described by the general term of e-commerce/e-business, can give new potential to the use of VRT in novel application domains. In fact VRT and Internet technologies can mutually benefit from ideas developed in the respective fields. This merging of technological potential can lead to a generalization of the concept of telework, where remote control through the network of actual physical processes will be possible. One can even think, for instance, of supervising and actively controlling a whole manufacturing process without having to move from his home. A major research objective must be of course to enable and promote new potential applications that can derive from the merging of such technologies, so that wider categories of the population can finally take benefit of these technological advances. At the rest of this chapter, we focus on analysing the theoretical foundations of this field and on describing the practical application domains that are involved, by presenting some characteristic case studies. Section 2 starts with a description of the basic principles related to virtual and augmented reality systems, and then presents an overview of applications related to the field of robotics. In Section 3 we describe the basic concepts that govern telerobotic systems and present a historical survey of the field. Section 4, then, presents typical application scenarios of these technologies, related to the two main robotic systems categories, namely robot manipulators and mobile robotic vehicles, and highlights the link with the new VR-based field of haptics. Concluding remarks and future research directions are given in Section 5. Virtual and Mixed Reality in Telerobotics: A Survey 439 Virtual and Mixed Reality: General Description During the last ten to fifteen years, Virtual Reality (VR), as a theoretical and applied research field, has attracted the interest of the international scientific community, as well as of the public opinion through extensive use of the term by the information, communication and entertainment media. However, the latter often results in an "abusive" use of this term, which is probably due to the lack of a formal definition of the field. In the sequel, we attempt to describe the basic principles that govern the field of VR, as well as of the more recent domain of Augmented and Mixed Reality systems, and we give an overview of related applications. Definitions and Basic Principles In an attempt to define what is a VR system, in relation to what can be seen as a simple human-computer or human-machine interaction, we can say that VR refers to: (a) computer generated and animated, three-dimensional realistic visualization space, enabling (b) real-time and multimodal interaction involving multiple sensori-motor channels of the human user, aiming to achieve (c) a sense of immersion and (virtual) presence in this synthetic (simulated) environment. The common factor here is, thus, the stimulation of human perceptual experience to produce an impression of something that does not really occur, but which is perceived and believed (potentially invoking, at some extent, human imagination) as being physically present and existing as a real world. The three important dimensions characterizing VR systems, and differentiating them from typical computer simulation environments, are: interaction, immersion and imagination, all contributing to create a sense of virtual presence and realism (Burdea & Coiffet, 94). A Virtual Environment (VE) created via graphics is a communication medium having both physical and abstract components. The three basic constituents of a VE are the content, the geometry and the dynamics (Ellis, 1995) . The content consists of objects and actors. The geometry is a description of the environmental field of action, and has dimensionality, metrics (rules establishing an ordering of the contents) and extent (range of possible values for the elements of the position vector). Dynamics is represented by the rules of interaction among the VE contents, describing their performance as they exchange information or energy. The components of a VE are useful for enhancing the interaction of the operators with their simulations. Virtualisation is defined to be the process by which an observer (viewer) interprets patterned sensory impressions to represent objects in environment other than that from which the impressions physically originate. Virtualisation can be applied to all senses: vision, audition, contact, shape and position (haptic sense). The three complementary technologies used to create the illusion of immersion in a VE are: Sensors (e.g. head position tracker or hand shape sensors) Effectors (e.g. stereoscopic displays or headphones) Special purpose hardware and software (connecting the sensors and effectors in such a way as to create experiences encountered by people immersed in a physical environment) A general diagram showing the structure of a VR-based system and the linkages of its components is shown in Fig. 1 . The human operator can interact with a VE presented by means of head and body referenced displays, the success depending on the fidelity with which sensory information is presented to the user. The environment experienced by the user via a VE simulation is of course imaginary. On the contrary, when referring to a 440 Industrial Robotics -Programming, Simulation and Applications
doi:10.5772/4911 fatcat:p5qhgw4lwrdsja7r4nj5e67pay