TY - JOUR
T1 - Quantifying Hypothesis Space Misspecification in Learning from Human-Robot Demonstrations and Physical Corrections
AU - Bobu, Andreea
AU - Bajcsy, Andrea
AU - Fisac, Jaime F.
AU - Deglurkar, Sampada
AU - Dragan, Anca D.
N1 - Funding Information:
Manuscript received April 20, 2019; accepted December 26, 2019. Date of publication February 24, 2020; date of current version June 4, 2020. This article was recommended for publication by Associate Editor Jun Morimoto and Editor A. Billard upon evaluation of the reviewers’ comments. This work was supported in part by the Air Force Office of Scientific Research and in part by the Open Philanthropy Project. (Corresponding author: Andreea Bobu.) A. Bobu is with the University of California, Berkeley, CA 94709 USA (e-mail: bobuandreea@gmail.com).
Funding Information:
Ms. Bajcsy was the recipient of the National Science Foundation’s Graduate Research Fellowship.
Publisher Copyright:
© 2004-2012 IEEE.
PY - 2020/6
Y1 - 2020/6
N2 - The human input has enabled autonomous systems to improve their capabilities and achieve complex behaviors that are otherwise challenging to generate automatically. Recent work focuses on how robots can use such inputs-such as, demonstrations or corrections-to learn intended objectives. These techniques assume that the human's desired objective already exists within the robot's hypothesis space. In reality, this assumption is often inaccurate: there will always be situations where the person might care about aspects of the task that the robot does not know about. Without this knowledge, the robot cannot infer the correct objective. Hence, when the robot's hypothesis space is misspecified, even methods that keep track of uncertainty over the objective fail because they reason about which hypothesis might be correct, and not whether any of the hypotheses are correct. In this article, we posit that the robot should reason explicitly about how well it can explain human inputs given its hypothesis space and use that situational confidence to inform how it should incorporate the human input. We demonstrate our method on a 7 degrees-of-freedom robot manipulator in learning from two important types of human inputs: demonstrations of motion planning tasks and physical corrections during the robot's task execution.
AB - The human input has enabled autonomous systems to improve their capabilities and achieve complex behaviors that are otherwise challenging to generate automatically. Recent work focuses on how robots can use such inputs-such as, demonstrations or corrections-to learn intended objectives. These techniques assume that the human's desired objective already exists within the robot's hypothesis space. In reality, this assumption is often inaccurate: there will always be situations where the person might care about aspects of the task that the robot does not know about. Without this knowledge, the robot cannot infer the correct objective. Hence, when the robot's hypothesis space is misspecified, even methods that keep track of uncertainty over the objective fail because they reason about which hypothesis might be correct, and not whether any of the hypotheses are correct. In this article, we posit that the robot should reason explicitly about how well it can explain human inputs given its hypothesis space and use that situational confidence to inform how it should incorporate the human input. We demonstrate our method on a 7 degrees-of-freedom robot manipulator in learning from two important types of human inputs: demonstrations of motion planning tasks and physical corrections during the robot's task execution.
KW - Bayesian inference
KW - inverse reinforcement learning (IRL)
KW - learning from demonstration
KW - physical human-robot interaction
UR - http://www.scopus.com/inward/record.url?scp=85087072419&partnerID=8YFLogxK
UR - http://www.scopus.com/inward/citedby.url?scp=85087072419&partnerID=8YFLogxK
U2 - 10.1109/TRO.2020.2971415
DO - 10.1109/TRO.2020.2971415
M3 - Article
AN - SCOPUS:85087072419
SN - 1552-3098
VL - 36
SP - 835
EP - 854
JO - IEEE Transactions on Robotics
JF - IEEE Transactions on Robotics
IS - 3
M1 - 9007490
ER -