This paper considers the application of reinforcement learning to path nding tasks in continuous state space in the presence of obstacles. We show that cumulative evaluation functions (as Q-Functions 28] and V-Functions 4]) may be discontinuous if forbidden regions (as implied by obstacles) exist in state space. As the in nite number of states requires the use of function approximators such as backpropagation nets 16, 12, 24], we argue that these discontinuities imply severe di culties in

more »

... ng cumulative evaluation functions. The discontinuities we detected might also explain why recent applications of reinforcement learning systems to complex tasks 12] failed to show desired performance. In our conclusion, we outline some ideas to circumvent the problem.