Vision-based closed-loop control of mobile microrobots for microhandling tasks

Axel Buerkle, Ferdinand Schmoeckel, Matthias Kiefer, Bala P. Amavasai, Fabio Caparrelli, Arul Selvan, Jon R. Travis, Bradley J. Nelson, Jean-Marc Breguet
2001 Microrobotics and Microassembly III  
As part of a European Union ESPRIT funded research project a flexible microrobot system has been developed which can operate under an optical microscope as well as in the chamber of a scanning electron microscope (SEM). The system is highly flexible and configurable and uses a wide range of sensors in a closed-loop control strategy. This paper presents an overview of the vision system and its architecture for vision-controlled micro-manipulation. The range of different applications, e.g.
more » ... ations, e.g. assembly of hybrid microsystems, handling of biological cells and manipulation tasks inside an SEM, imposes great demands on the vision system. Fast and reliable object recognition algorithms have been developed and implemented to provide for two modes of operation: automated and semi-automated robot control. The vision system has a modular design, comprising modules for object recognition, tracking and depth estimation. Communication between the vision modules and the control system takes place via a shared memory system embedding an object database. This database holds information about the appearance and the location of all known objects. A depth estimation method based on a modified sheet-of-light triangulation method is also described. It is fully integrated in the control loop and can be used both for measuring specific points or scanning a complete field of view. Furthermore, the novel approach of electron beam triangulation in the SEM is described. To ensure co-ordinated operation between the vision subsystem and the microrobot controller, an interprocess communication (IPC) system has been implemented. IPC takes place through shared memory using semaphore control. This allows for bi-directional flow of information data. Figure 2 shows a block diagram of the IPC system. To install a communication channel, either the vision subsystem or the microrobot controller is first initiated and upon the successful creation of the IPC link, set into wait mode. Operations may only take place once both the vision subsystem and the microrobot controller are launched. Three shared memory segments are allocated, one for the predefined object database, one for the scene images and one for objects located in the scene.
doi:10.1117/12.444125 fatcat:mfh7avdnpfedlpos2lmbqtt3ze