TY - GEN
T1 - An intelligent battery controller using bias-corrected Q-learning
AU - Lee, Donghun
AU - Powell, Warren Buckler
PY - 2012
Y1 - 2012
N2 - The transition to renewables requires storage to help smooth short-term variations in energy from wind and solar sources, as well as to respond to spikes in electricity spot prices, which can easily exceed 20 times their average. Efficient operation of an energy storage device is a fundamental problem, yet classical algorithms such as Q-learning can diverge for millions of iterations, limiting practical applications. We have traced this behavior to the max-operator bias, which is exacerbated by high volatility in the reward function, and high discount factors due to the small time steps. We propose an elegant bias correction procedure and demonstrate its effectiveness.
AB - The transition to renewables requires storage to help smooth short-term variations in energy from wind and solar sources, as well as to respond to spikes in electricity spot prices, which can easily exceed 20 times their average. Efficient operation of an energy storage device is a fundamental problem, yet classical algorithms such as Q-learning can diverge for millions of iterations, limiting practical applications. We have traced this behavior to the max-operator bias, which is exacerbated by high volatility in the reward function, and high discount factors due to the small time steps. We propose an elegant bias correction procedure and demonstrate its effectiveness.
UR - http://www.scopus.com/inward/record.url?scp=84868293162&partnerID=8YFLogxK
UR - http://www.scopus.com/inward/citedby.url?scp=84868293162&partnerID=8YFLogxK
M3 - Conference contribution
AN - SCOPUS:84868293162
SN - 9781577355687
T3 - Proceedings of the National Conference on Artificial Intelligence
SP - 316
EP - 322
BT - AAAI-12 / IAAI-12 - Proceedings of the 26th AAAI Conference on Artificial Intelligence and the 24th Innovative Applications of Artificial Intelligence Conference
T2 - 26th AAAI Conference on Artificial Intelligence and the 24th Innovative Applications of Artificial Intelligence Conference, AAAI-12 / IAAI-12
Y2 - 22 July 2012 through 26 July 2012
ER -