A Bayesian network approach to control of networked Markov decision processes

Sachin Adlakha, Sanjay Lall, Andrea Goldsmith

Research output: Chapter in Book/Report/Conference proceedingConference contribution

4 Scopus citations

Abstract

We consider the problem of finding an optimal feedback controller for a networked Markov decision process. Specifically, we consider a network of interconnected subsystems, where each subsystem evolves as a Markov decision process (MDP). A subsystem is connected to its neighbors via links over which signals are delayed. We consider centralized control of such networked MDPs. The controller receives delayed state information from each of the subsystem, and it chooses control actions for all subsystems. Such networked MDPs can be represented as partially observed Markov decision processes (POMDPs). We model such a POMDP as a Bayesian network and show that an optimal controller requires only a finite history of past states and control actions. The result is based on the idea that given certain past states and actions, the current state of the networked MDP is independent of the earlier states and actions. This dependence on only the finite past states and actions makes the computation of controllers for networked MDPs tractable.

Original languageEnglish (US)
Title of host publication46th Annual Allerton Conference on Communication, Control, and Computing
Pages446-451
Number of pages6
DOIs
StatePublished - 2008
Externally publishedYes
Event46th Annual Allerton Conference on Communication, Control, and Computing - Monticello, IL, United States
Duration: Sep 24 2008Sep 26 2008

Publication series

Name46th Annual Allerton Conference on Communication, Control, and Computing

Other

Other46th Annual Allerton Conference on Communication, Control, and Computing
CountryUnited States
CityMonticello, IL
Period9/24/089/26/08

All Science Journal Classification (ASJC) codes

  • Computer Networks and Communications
  • Software
  • Control and Systems Engineering
  • Communication

Fingerprint Dive into the research topics of 'A Bayesian network approach to control of networked Markov decision processes'. Together they form a unique fingerprint.

Cite this