TY - JOUR
T1 - Reinforcement learning for electricity dispatch in grids with high intermittent generation and energy storage systems
T2 - A case study for the Brazilian grid
AU - de Carvalho Neiva Pinheiro, Vinícius
AU - Francato, Alberto L.
AU - Powell, Warren B.
N1 - Funding Information:
The authors would like to thank CAPES—Brazilian Federal Agency for Support and Evaluation of Graduate Education—for funding this work.
Publisher Copyright:
© 2020 John Wiley & Sons Ltd
PY - 2020/9/1
Y1 - 2020/9/1
N2 - Intermittent energy sources such as wind and solar have recently been growing a lot faster than dispatchable energy sources in Brazil, which made investments in energy storage systems become an attractive possibility in the country. Current operational policies for energy dispatch do not consider storage systems and need adjustments to fit this technology. With this motivation, we use reinforcement learning techniques to develop policies for managing storage systems in a grid that can handle time-varying inputs and loads, with rolling forecasts. We use a deterministic lookahead (DLA) policy which has been parametrically modified to perform well in the presence of uncertain forecasts. For realistic simulations, the base model considers important characteristics in a grid that influence the interaction between scheduling and real-time operation such as power and ramping capacities, notification times, and stochastic forecasts. The parametric modification with tunable parameters allows an optimal balance between two conflicting services provided by the storage system: time-shifting and spinning reserves. Optimal reserves ranged from 35% to 100%, depending on the tested dataset, which shows the importance of tuning. Differently from stochastic lookahead policies, which are computationally expensive, parameterized DLA policies can be applied to real-time operation after being optimized in a stochastic base model.
AB - Intermittent energy sources such as wind and solar have recently been growing a lot faster than dispatchable energy sources in Brazil, which made investments in energy storage systems become an attractive possibility in the country. Current operational policies for energy dispatch do not consider storage systems and need adjustments to fit this technology. With this motivation, we use reinforcement learning techniques to develop policies for managing storage systems in a grid that can handle time-varying inputs and loads, with rolling forecasts. We use a deterministic lookahead (DLA) policy which has been parametrically modified to perform well in the presence of uncertain forecasts. For realistic simulations, the base model considers important characteristics in a grid that influence the interaction between scheduling and real-time operation such as power and ramping capacities, notification times, and stochastic forecasts. The parametric modification with tunable parameters allows an optimal balance between two conflicting services provided by the storage system: time-shifting and spinning reserves. Optimal reserves ranged from 35% to 100%, depending on the tested dataset, which shows the importance of tuning. Differently from stochastic lookahead policies, which are computationally expensive, parameterized DLA policies can be applied to real-time operation after being optimized in a stochastic base model.
KW - energy storage
KW - intermittent energy sources
KW - operational policies
KW - policy design
KW - stochastic forecast
UR - http://www.scopus.com/inward/record.url?scp=85085885411&partnerID=8YFLogxK
UR - http://www.scopus.com/inward/citedby.url?scp=85085885411&partnerID=8YFLogxK
U2 - 10.1002/er.5551
DO - 10.1002/er.5551
M3 - Article
AN - SCOPUS:85085885411
SN - 0363-907X
VL - 44
SP - 8635
EP - 8653
JO - International Journal of Energy Research
JF - International Journal of Energy Research
IS - 11
ER -