Learning Task-Driven Control Policies via Information Bottlenecks

Vincent Pacelli, Anirudha Majumdar

Research output: Chapter in Book/Report/Conference proceedingConference contribution

1 Scopus citations

Abstract

This paper presents a reinforcement learning approach to synthesizing task-driven control policies for robotic systems equipped with rich sensory modalities (e.g., vision or depth). Standard reinforcement learning algorithms typically produce policies that tightly couple control actions to the entirety of the system’s state and rich sensor observations. As a consequence, the resulting policies can often be sensitive to changes in task-irrelevant portions of the state or observations (e.g., changing background colors). In contrast, the approach we present here learns to create a task-driven representation that is used to compute control actions. Formally, this is achieved by deriving a policy gradient-style algorithm that creates an information bottleneck between the states and the task-driven representation; this constrains actions to only depend on task-relevant information. We demonstrate our approach in a thorough set of simulation results on multiple examples including a grasping task that utilizes depth images and a ball-catching task that utilizes RGB images. Comparisons with a standard policy gradient approach demonstrate that the task-driven policies produced by our algorithm are often significantly more robust to sensor noise and task-irrelevant changes in the environment.

Original languageEnglish (US)
Title of host publicationRobotics
Subtitle of host publicationScience and Systems XVI
EditorsMarc Toussaint, Antonio Bicchi, Tucker Hermans
PublisherMIT Press Journals
ISBN (Print)9780992374761
DOIs
StatePublished - 2020
Event16th Robotics: Science and Systems, RSS 2020 - Virtual, Online
Duration: Jul 12 2020Jul 16 2020

Publication series

NameRobotics: Science and Systems
ISSN (Electronic)2330-765X

Conference

Conference16th Robotics: Science and Systems, RSS 2020
CityVirtual, Online
Period7/12/207/16/20

All Science Journal Classification (ASJC) codes

  • Artificial Intelligence
  • Control and Systems Engineering
  • Electrical and Electronic Engineering

Fingerprint

Dive into the research topics of 'Learning Task-Driven Control Policies via Information Bottlenecks'. Together they form a unique fingerprint.

Cite this