Designing individual agents so that, when put together, they reach a given global goal is not an easy task. One solution to automatically build such large Multi-Agent Systems is to use decentralized learning : each agent learns by itself its own behavior. To that purpose, Reinforcement Learning methods are very attractive as they do not require a solution of the problem to be known before hand. Nevertheless, many hard points need to be solved for such a learning process to be viable. Among

more »

... s, the credit assignement problem, combinatorial explosion and local perception of the world seem the most crucial and prevent optimal behavior. In this paper, we propose a framework based on a gradual learning of harder and harder tasks until the desired global behavior is reached. The applicability of our paradigm is tested on computer experiments where many agents have to coordinate to reach a global goal. Our results show that incremental learning leads to better performances than more classical techniques. We then discuss several improvements which could lead to even better performances.