Probably Approximately Correct Vision-Based Planning using Motion Primitives

Sushant Veer, Anirudha Majumdar

Research output: Contribution to journalConference articlepeer-review

6 Scopus citations


This paper presents an approach for learning vision-based planners that provably generalize to novel environments (i.e., environments unseen during training). We leverage the Probably Approximately Correct (PAC)-Bayes framework to obtain an upper bound on the expected cost of policies across all environments. Minimizing the PAC-Bayes upper bound thus trains policies that are accompanied by a certificate of performance on novel environments. The training pipeline we propose provides strong generalization guarantees for deep neural network policies by (a) obtaining a good prior distribution on the space of policies using Evolutionary Strategies (ES) followed by (b) formulating the PAC-Bayes optimization as an efficiently-solvable parametric convex optimization problem. We demonstrate the efficacy of our approach for producing strong generalization guarantees for learned vision-based motion planners through two simulated examples: (1) an Unmanned Aerial Vehicle (UAV) navigating obstacle fields with an onboard vision sensor, and (2) a dynamic quadrupedal robot traversing rough terrains with proprioceptive and exteroceptive sensors.

Original languageEnglish (US)
Pages (from-to)1001-1014
Number of pages14
JournalProceedings of Machine Learning Research
StatePublished - 2020
Externally publishedYes
Event4th Conference on Robot Learning, CoRL 2020 - Virtual, Online, United States
Duration: Nov 16 2020Nov 18 2020

All Science Journal Classification (ASJC) codes

  • Artificial Intelligence
  • Software
  • Control and Systems Engineering
  • Statistics and Probability


  • PAC-Bayes
  • generalization
  • vision-based motion planning


Dive into the research topics of 'Probably Approximately Correct Vision-Based Planning using Motion Primitives'. Together they form a unique fingerprint.

Cite this