Regret based dynamics

Jason R. Marden, Gürdal Arslan, Jeff S. Shamma
2007 Proceedings of the 6th international joint conference on Autonomous agents and multiagent systems - AAMAS '07  
No-regret algorithms have been proposed to control a wide variety of multi-agent systems. The appeal of no-regret algorithms is that they are easily implementable in large scale multi-agent systems because players make decisions using only retrospective or "regret based" information. Furthermore, there are existing results proving that the collective behavior will asymptotically converge to a set of points of "no-regret" in any game. We illustrate, through a simple example, that no-regret
more » ... need not reflect desirable operating conditions for a multi-agent system. Multi-agent systems often exhibit an additional structure (i.e. being "weakly acyclic") that has not been exploited in the context of no-regret algorithms. In this paper, we introduce a modification of the traditional no-regret algorithms by (i) exponentially discounting the memory and (ii) bringing in a notion of inertia in players' decision process. We show how these modifications can lead to an entire class of regret based algorithms that provide almost sure convergence to a pure Nash equilibrium in any weakly acyclic game.
doi:10.1145/1329125.1329175 dblp:conf/atal/MardenAS07 fatcat:clfa3cklmnebfkvy4ihlycefg4