An intelligent battery controller using bias-corrected Q-learning

Donghun Lee, Warren Buckler Powell

Research output: Chapter in Book/Report/Conference proceedingConference contribution

17 Scopus citations

Abstract

The transition to renewables requires storage to help smooth short-term variations in energy from wind and solar sources, as well as to respond to spikes in electricity spot prices, which can easily exceed 20 times their average. Efficient operation of an energy storage device is a fundamental problem, yet classical algorithms such as Q-learning can diverge for millions of iterations, limiting practical applications. We have traced this behavior to the max-operator bias, which is exacerbated by high volatility in the reward function, and high discount factors due to the small time steps. We propose an elegant bias correction procedure and demonstrate its effectiveness.

Original languageEnglish (US)
Title of host publicationAAAI-12 / IAAI-12 - Proceedings of the 26th AAAI Conference on Artificial Intelligence and the 24th Innovative Applications of Artificial Intelligence Conference
Pages316-322
Number of pages7
StatePublished - 2012
Event26th AAAI Conference on Artificial Intelligence and the 24th Innovative Applications of Artificial Intelligence Conference, AAAI-12 / IAAI-12 - Toronto, ON, Canada
Duration: Jul 22 2012Jul 26 2012

Publication series

NameProceedings of the National Conference on Artificial Intelligence
Volume1

Other

Other26th AAAI Conference on Artificial Intelligence and the 24th Innovative Applications of Artificial Intelligence Conference, AAAI-12 / IAAI-12
Country/TerritoryCanada
CityToronto, ON
Period7/22/127/26/12

All Science Journal Classification (ASJC) codes

  • Software
  • Artificial Intelligence

Fingerprint

Dive into the research topics of 'An intelligent battery controller using bias-corrected Q-learning'. Together they form a unique fingerprint.

Cite this