A note on the representational incompatibility of function approximation and factored dynamics

Eric Allender, Sanjeev Arora, Michael Kearns, Cristopher Moore, Alexander Russell

Research output: Chapter in Book/Report/Conference proceedingConference contribution

1 Scopus citations

Abstract

We establish a new hardness result that shows that the difficulty of planning in factored Markov decision processes is representational rather than just computational. More precisely, we give & fixed family of factored MDPs with linear rewards whose optimal policies and value functions simply cannot be represented succinctly in any standard parametric form. Previous hardness results indicated that computing good policies from the MDP parameters was difficult, but left open the possibility of succinct function approximation for any fixed factored MDP. Our result applies even to policies which yield a polynomially poor approximation to the optimal value, and highlights interesting connections with the complexity class of Arthur-Merlin games.

Original languageEnglish (US)
Title of host publicationAdvances in Neural Information Processing Systems 15 - Proceedings of the 2002 Conference, NIPS 2002
PublisherNeural information processing systems foundation
ISBN (Print)0262025507, 9780262025508
StatePublished - Jan 1 2003
Event16th Annual Neural Information Processing Systems Conference, NIPS 2002 - Vancouver, BC, Canada
Duration: Dec 9 2002Dec 14 2002

Publication series

NameAdvances in Neural Information Processing Systems
ISSN (Print)1049-5258

Other

Other16th Annual Neural Information Processing Systems Conference, NIPS 2002
CountryCanada
CityVancouver, BC
Period12/9/0212/14/02

All Science Journal Classification (ASJC) codes

  • Computer Networks and Communications
  • Information Systems
  • Signal Processing

Fingerprint Dive into the research topics of 'A note on the representational incompatibility of function approximation and factored dynamics'. Together they form a unique fingerprint.

  • Cite this

    Allender, E., Arora, S., Kearns, M., Moore, C., & Russell, A. (2003). A note on the representational incompatibility of function approximation and factored dynamics. In Advances in Neural Information Processing Systems 15 - Proceedings of the 2002 Conference, NIPS 2002 (Advances in Neural Information Processing Systems). Neural information processing systems foundation.