Reachability-based safe learning with Gaussian processes

Anayo K. Akametalu, Shahab Kaynama, Jaime F. Fisac, Melanie N. Zeilinger, Jeremy H. Gillula, Claire J. Tomlin

Research output: Chapter in Book/Report/Conference proceedingConference contribution

128 Scopus citations

Abstract

Reinforcement learning for robotic applications faces the challenge of constraint satisfaction, which currently impedes its application to safety critical systems. Recent approaches successfully introduce safety based on reachability analysis, determining a safe region of the state space where the system can operate. However, overly constraining the freedom of the system can negatively affect performance, while attempting to learn less conservative safety constraints might fail to preserve safety if the learned constraints are inaccurate. We propose a novel method that uses a principled approach to learn the system's unknown dynamics based on a Gaussian process model and iteratively approximates the maximal safe set. A modified control strategy based on real-time model validation preserves safety under weaker conditions than current approaches. Our framework further incorporates safety into the reinforcement learning performance metric, allowing a better integration of safety and learning. We demonstrate our algorithm on simulations of a cart-pole system and on an experimental quadrotor application and show how our proposed scheme succeeds in preserving safety where current approaches fail to avoid an unsafe condition.

Original languageEnglish (US)
Title of host publication53rd IEEE Conference on Decision and Control,CDC 2014
PublisherInstitute of Electrical and Electronics Engineers Inc.
Pages1424-1431
Number of pages8
EditionFebruary
ISBN (Electronic)9781479977468
DOIs
StatePublished - 2014
Externally publishedYes
Event2014 53rd IEEE Annual Conference on Decision and Control, CDC 2014 - Los Angeles, United States
Duration: Dec 15 2014Dec 17 2014

Publication series

NameProceedings of the IEEE Conference on Decision and Control
NumberFebruary
Volume2015-February
ISSN (Print)0743-1546
ISSN (Electronic)2576-2370

Other

Other2014 53rd IEEE Annual Conference on Decision and Control, CDC 2014
Country/TerritoryUnited States
CityLos Angeles
Period12/15/1412/17/14

All Science Journal Classification (ASJC) codes

  • Control and Systems Engineering
  • Modeling and Simulation
  • Control and Optimization

Fingerprint

Dive into the research topics of 'Reachability-based safe learning with Gaussian processes'. Together they form a unique fingerprint.

Cite this