Exploring the Limitations of Behavior Cloning for Autonomous Driving

Felipe Codevilla, Eder Santana, Antonio Lopez, Adrien Gaidon
2019 2019 IEEE/CVF International Conference on Computer Vision (ICCV)  
Figure 1 . Driving scenarios from our new benchmark, based on the CARLA simulator, where the agent needs to react to dynamic changes in the environment, handle clutter (only part of the environment is causally relevant), and predict complex sensorimotor controls (lateral and longitudinal). We show that Behavior Cloning yields state-of-the-art policies in these complex scenarios and investigate its limitations. Abstract Driving requires reacting to a wide variety of complex environment
more » ... and agent behaviors. Explicitly modeling each possible scenario is unrealistic. In contrast, imitation learning can, in theory, leverage data from large fleets of human-driven cars. Behavior cloning in particular has been successfully used to learn simple visuomotor policies end-to-end, but scaling to the full spectrum of driving behaviors remains an unsolved problem. In this paper, we propose a new benchmark to experimentally investigate the scalability and limitations of behavior cloning. We show that behavior cloning leads to state-ofthe-art results, executing complex lateral and longitudinal maneuvers, even in unseen environments, without being explicitly programmed to do so. However, we confirm some limitations of the behavior cloning approach: some wellknown limitations (e.g., dataset bias and overfitting), new generalization issues (e.g., dynamic objects and the lack of a causal modeling), and training instabilities, all requiring further research before behavior cloning can graduate to real-world driving. The code, dataset, benchmark, and agent studied in this paper can be found at
doi:10.1109/iccv.2019.00942 dblp:conf/iccv/CodevillaSLG19 fatcat:sbmedxvpwnhjtk6oajprvnw6mi