Deep reinforcement learning control of electric vehicle charging in the presence of photovoltaic generation

Marina Dorokhova, Yann Martinson, Christophe Ballif, Nicolas Wyrsch
2021 Applied Energy  
A B S T R A C T In recent years, the importance of electric mobility has increased in response to climate change. The fastgrowing deployment of electric vehicles (EVs) worldwide is expected to decrease transportation-related 2 emissions, facilitate the integration of renewables, and support the grid through demand-response services. Simultaneously, inadequate EV charging patterns can lead to undesirable effects in grid operation, such as high peak-loads or low self-consumption of solar
more » ... ty, thus calling for novel methods of control. This work focuses on applying deep reinforcement learning (RL) to the EV charging control problem with the objectives to increase photovoltaic self-consumption and EV state of charge at departure. Particularly, we propose mathematical formulations of environments with discrete, continuous, and parametrized action spaces and respective deep RL algorithms to resolve them. The benchmarking of the deep RL control against naive, rule-based, deterministic optimization, and model-predictive control demonstrates that the suggested methodology can produce consistent and employable EV charging strategies, while its performance holds a great promise for real-time implementations. (M. Dorokhova). the EVs are parked more than 80% of the time [3], which gives the potential to intelligently shift the charging load, thus deploying smart energy management techniques. On the bright side of the increasing penetration of electric mobility is the opportunity to offer grid ancillary services to support the grid's various objectives. For example, using EVs can reduce energy costs, contribute to peak shaving, improve system balancing, and integrate a larger share of renewables into power production. However, the trade-off is to combine demand-response with seamless availability of EVs, such as the primary purpose of enabling mobility is served in a reliable and timely manner. Control methods To effectively manage the charging processes of EVs, one has to choose between various control strategies. The three main broad classes of control methods include rule-based control (RBC), model predictive control (MPC), and reinforcement learning control (RLC) [4] . Indeed, there are certain advantages and disadvantages associated with each of these control techniques. Therefore, one has to choose an appropriate approach based on the trade-off between the complexity of the problem, control objectives, and available computational resources.
doi:10.1016/j.apenergy.2021.117504 fatcat:sjdvhlj25rhdxalgxi4dia7b7q