Abstract
Robust and generalized tool manipulation requires an understanding of the properties and affordances of different tools. We investigate whether linguistic information about a tool (e.g., its geometry, common uses) can help control policies adapt faster to new tools for a given task. We obtain diverse descriptions of various tools in natural language and use pre-trained language models to generate their feature representations. We then perform language-conditioned meta-learning to learn policies that can efficiently adapt to new tools given their corresponding text descriptions. Our results demonstrate that combining linguistic information and meta-learning significantly accelerates tool learning in several manipulation tasks including pushing, lifting, sweeping, and hammering.
Original language | English (US) |
---|---|
Pages (from-to) | 1531-1541 |
Number of pages | 11 |
Journal | Proceedings of Machine Learning Research |
Volume | 205 |
State | Published - 2023 |
Externally published | Yes |
Event | 6th Conference on Robot Learning, CoRL 2022 - Auckland, New Zealand Duration: Dec 14 2022 → Dec 18 2022 |
All Science Journal Classification (ASJC) codes
- Artificial Intelligence
- Software
- Control and Systems Engineering
- Statistics and Probability
Keywords
- Language for Robotics
- Meta-learning
- Tool Manipulation