Learning from Interventions: Human-robot interaction as both explicit and implicit feedback

Jonathan Spencer, Sanjiban Choudhury, Matthew Barnes, Matthew Schmittle, Mung Chiang, Peter Ramadge, Siddhartha Srinivasa

Research output: Chapter in Book/Report/Conference proceedingConference contribution

21 Scopus citations

Abstract

Scalable robot learning from seamless human-robot interaction is critical if robots are to solve a multitude of tasks in the real world. Current approaches to imitation learning suffer from one of two drawbacks. On the one hand, they rely solely on off-policy human demonstration, which in some cases leads to a mismatch in train-test distribution. On the other, they burden the human to label every state the learner visits, rendering it impractical in many applications. We argue that learning interactively from expert interventions enjoys the best of both worlds. Our key insight is that any amount of expert feedback, whether by intervention or non-intervention, provides information about the quality of the current state, the optimality of the action, or both. We formalize this as a constraint on the learner’s value function, which we can efficiently learn using no regret, online learning techniques. We call our approach Expert Intervention Learning (EIL), and evaluate it on a real and simulated driving task with a human expert, where it learns collision avoidance from scratch with just a few hundred samples (about one minute) of expert control.

Original languageEnglish (US)
Title of host publicationRobotics
Subtitle of host publicationScience and Systems XVI
EditorsMarc Toussaint, Antonio Bicchi, Tucker Hermans
PublisherMIT Press Journals
ISBN (Print)9780992374761
DOIs
StatePublished - 2020
Event16th Robotics: Science and Systems, RSS 2020 - Virtual, Online
Duration: Jul 12 2020Jul 16 2020

Publication series

NameRobotics: Science and Systems
ISSN (Electronic)2330-765X

Conference

Conference16th Robotics: Science and Systems, RSS 2020
CityVirtual, Online
Period7/12/207/16/20

All Science Journal Classification (ASJC) codes

  • Artificial Intelligence
  • Control and Systems Engineering
  • Electrical and Electronic Engineering

Fingerprint

Dive into the research topics of 'Learning from Interventions: Human-robot interaction as both explicit and implicit feedback'. Together they form a unique fingerprint.

Cite this