Scalable Voltage Control using Structure-Driven Hierarchical Deep Reinforcement Learning [article]

Sayak Mukherjee, Renke Huang, Qiuhua Huang, Thanh Long Vu, Tianzhixi Yin
<span title="2021-01-29">2021</span> <i > arXiv </i> &nbsp; <span class="release-stage" >pre-print</span>
This paper presents a novel hierarchical deep reinforcement learning (DRL) based design for the voltage control of power grids. DRL agents are trained for fast, and adaptive selection of control actions such that the voltage recovery criterion can be met following disturbances. Existing voltage control techniques suffer from the issues of speed of operation, optimal coordination between different locations, and scalability. We exploit the area-wise division structure of the power system to
more &raquo; ... se a hierarchical DRL design that can be scaled to the larger grid models. We employ an enhanced augmented random search algorithm that is tailored for the voltage control problem in a two-level architecture. We train area-wise decentralized RL agents to compute lower-level policies for the individual areas, and concurrently train a higher-level DRL agent that uses the updates of the lower-level policies to efficiently coordinate the control actions taken by the lower-level agents. Numerical experiments on the IEEE benchmark 39-bus model with 3 areas demonstrate the advantages and various intricacies of the proposed hierarchical approach.
<span class="external-identifiers"> <a target="_blank" rel="external noopener" href="https://arxiv.org/abs/2102.00077v1">arXiv:2102.00077v1</a> <a target="_blank" rel="external noopener" href="https://fatcat.wiki/release/lcme6ztwpne5vbn2za2hmremii">fatcat:lcme6ztwpne5vbn2za2hmremii</a> </span>
<a target="_blank" rel="noopener" href="https://web.archive.org/web/20210216164503/https://arxiv.org/pdf/2102.00077v1.pdf" title="fulltext PDF download" data-goatcounter-click="serp-fulltext" data-goatcounter-title="serp-fulltext"> <button class="ui simple right pointing dropdown compact black labeled icon button serp-button"> <i class="icon ia-icon"></i> Web Archive [PDF] <div class="menu fulltext-thumbnail"> <img src="https://blobs.fatcat.wiki/thumbnail/pdf/b3/c2/b3c29068f2fe844c8c6116a220ff24733d8620dd.180px.jpg" alt="fulltext thumbnail" loading="lazy"> </div> </button> </a> <a target="_blank" rel="external noopener" href="https://arxiv.org/abs/2102.00077v1" title="arxiv.org access"> <button class="ui compact blue labeled icon button serp-button"> <i class="file alternate outline icon"></i> arxiv.org </button> </a>