Analyzing Stigmergetic Algorithms Through Automata Games [chapter]

Peter Vrancx, Katja Verbeeck, Ann Nowé
Knowledge Discovery and Emergent Complexity in Bioinformatics  
The concept of stigmergy is usually defined as a class of mechanisms that mediate animal to animal interaction through the environment. The idea behind stigmergy is that individuals coordinate their actions by locally modifying the environment rather than by direct interaction. The changed environmental situation caused by one animal, will stimulate others to perform certain actions. This concept has been used to explain the coordinated behavior of social insects such as termites, ants and
more » ... Recently the notion of stigmergy has gained interest in the domains of multiagent systems and agent based computing Algorithms such as Ant Colony Optimization (ACO) model aspects of social insect behavior to coordinate agent behavior and cooperation. The concept of stigmergy is promising in this context, as it provides a relatively simple framework for agent communication and coordination. One of the main problems that arises, however, is the difficulty of determining the global system behavior that will arise from local stigmergetic interactions. We propose to model stigmergetic communication by networks of learning automata (LA). One of the main advantages of LA is the existence of convergence proofs for many possible interaction schemes. Using LA as a model for stigmergy allows us to transfer some of these insights to stigmergetic algorithms. One principal and useful contribution of LA theory is that a set of decentralized learning automata is able to control a finite Markov Chain with unknown transition probabilities and rewards. Applying our LA-model will allow us to use existing automata theory in order to analyze stigmergetic algorithms in terms of their global behaviour. Using the Markov Chain result mentioned above we can map the stigmergetic algorithm to a game of learning automata. In case of simple single type pheromone updates with agents sharing a common goal, as in the ACO model, convergence is guarenteed to a global optimum as long as the underlying network satisfies some ergodic assumption. In case of multiple pheromones and multiple agent goals, convergence is still assured to a (possibly) suboptimal attractor point.
doi:10.1007/978-3-540-71037-0_10 dblp:conf/kdecb/VrancxVN06 fatcat:2tpnmz522jdufoobphpzpt7nda