An Optimal Approximate Dynamic Programming Algorithm for the Lagged Asset Acquisition Problem
Mathematics of Operations Research
We prove convergence of an approximate dynamic programming algorithm for a class of high-dimensional stochastic control problems linked by a scalar storage device, given a technical condition. Our problem is motivated by the problem of optimizing energy flows for a power grid supported by grid-level storage. The problem is formulated as a stochastic, dynamic program, where we estimate the value of resources in storage using a piecewise linear value function approximation. Given the technical
... dition, we provide a rigorous convergence proof for an approximate dynamic programming algorithm, which can capture the presence of both the amount of energy held in storage as well as other exogenous variables. Our algorithm exploits the natural concavity of the problem to avoid any need for explicit exploration policies. Index Terms-Approximate dynamic programming, resource allocation, storage. Juliana Nascimento received the Ph.D. degree in Operations Research and Financial Engineering from Princeton University, Princeton, NJ, USA. After finishing her Ph.D., she joined McKinsey & Company, São Paulo, Brazil. Currently she works at Kimberly-Clark Brazil, São Paulo, and is in charge of Planning, International trade and Projects. She has developed and implemented a portfolio optimization project in the company and is currently working at a resource allocation/demand management project that considers the fiscal environment in Brazil. Warren B. Powell is currently a professor in the Department of Operations Research and Financial Engineering, Princeton University, Princeton, NJ, USA, where he has taught since 1981. His research specializes in stochastic optimization, with applications in energy, transportation, health, and finance. He has authored/coauthored over 190 publications and two books. He founded and directs the CASTLE Laboratory and the Princeton Laboratory for Energy Systems Analysis (PENSA).