Cognitive Systems Platforms using Open Source
Performance Evaluation and Benchmarking of Intelligent Systems
This chapter reports to the development of the tools and methodologies that are in development within the EU, with an emphasis on the Open Source approaches with a view to performance analysis and comparison, and to provide an overview of cooperative research and especially on the use of Open platforms. The Rats Life Benchmark Rat's Life is a complete cognitive robotics benchmark that was carefully designed to be easily reproducible in a research lab with limited resources. It relies on two
... ck robots, some LEGO bricks and the Webots robot simulation software. This benchmark is a survival game where two robots compete against each other for resources in a previously unseen maze. Like the rats in cognitive animal experimentation, the epuck robots look for feeders which allow them to live longer than their opponent. Once a feeder is reached by a robot, the robot draws energy from it and the feeder becomes unavailable for a while. Hence, the robot has to further explore the maze, searching for other feeders. The iCub platform The iCub is a new Open Source humanoid robot developed for research in embodied cognition. At around the size of a three and half year old child, it can crawl on all fours and sit up to manipulate objects. Its hands have been designed to support sophisticate manipulation skills. The iCub is distributed as Open Source following the GPL/FDL licenses. The entire design is available for download from the project homepage and repository 2 . In the following, we concentrate on the description of the hardware and software systems, as well as an iCub simulator, each of which may be used in comparative studies. The swarm platform of the Replicator and Symbrion projects Cooperation and competition among stand-alone swarm agents can increase the collective fitness of the whole system. An interesting form of collective system is demonstrated by some bacteria and fungi, which can build symbiotic organisms. Symbiotic communities can enable new functional capabilities which allow all members to survive better in their environment. Here we present an overview of two large European projects dealing with new collective robotic systems which utilize principles derived from natural symbiosis. We also describe some of the typical hardware, software and methodological challenges arising, as well as prototypes and on-going experiments. The use of multiple robotic agents lends itself to the replication of experiments and thus benchmarking of behavior and functionality. 2 The Rat's Life Benchmark: Competing Cognitive Robots 2.1 Motivation Most scientific publications in the area of robotics research face tremendous challenges: comparing the achieved result with other similar research results and hence convincing the reader of the quality of the research work. These challenges are very 3 EURON Benchmarking. www.euron.org/activities/benchmarks/index.html Robot Benchmark Euron website www.robot.uji.es/EURON/en/euron07.htm NIST ISD website http://www.isd.mel.nist.gov/ Euron GEM SIG website http://www.heronrobots.com/EuronGEMSig/ Rawseeds web site http://www.rawseeds.org/ RoSta website http://www.robot-standards.eu/ 4 Micromouse contest. http://www.micromouseinfo.com 5 AAAI Robot Competition and exhibition.