TY - GEN
T1 - Look and Listen
T2 - 39th IEEE International Conference on Robotics and Automation, ICRA 2022
AU - Burns, Alexis
AU - Xiang, Siyuan
AU - Lee, Daewon
AU - Jackel, Larry
AU - Song, Shuran
AU - Isler, Volkan
N1 - Publisher Copyright:
© 2022 IEEE.
PY - 2022
Y1 - 2022
N2 - Humans have the ability to pour various media, both liquid and granular, to desired ends in various containers. We do this by using multiple senses simultaneously in a constant feedback loop to complete a pouring task. Combining multiple sensing modalities, similar to humans, could aid in robotic pouring control outside of a structured or industrial setting. We present a multi-sensory pouring dataset consisting of human pouring demonstrations of various granular media, coupled with two multi-sensory networks that estimate pouring rate and pouring average height. For both pouring metrics, a combined input of audio and visual data provides a lower median error than either the audio network or visual network. The multi-sensory network achieves a median error of 6.4 mm for average height estimation and 0.06 N/s for pouring rate estimation.
AB - Humans have the ability to pour various media, both liquid and granular, to desired ends in various containers. We do this by using multiple senses simultaneously in a constant feedback loop to complete a pouring task. Combining multiple sensing modalities, similar to humans, could aid in robotic pouring control outside of a structured or industrial setting. We present a multi-sensory pouring dataset consisting of human pouring demonstrations of various granular media, coupled with two multi-sensory networks that estimate pouring rate and pouring average height. For both pouring metrics, a combined input of audio and visual data provides a lower median error than either the audio network or visual network. The multi-sensory network achieves a median error of 6.4 mm for average height estimation and 0.06 N/s for pouring rate estimation.
UR - https://www.scopus.com/pages/publications/85136330529
UR - https://www.scopus.com/pages/publications/85136330529#tab=citedBy
U2 - 10.1109/ICRA46639.2022.9812125
DO - 10.1109/ICRA46639.2022.9812125
M3 - Conference contribution
AN - SCOPUS:85136330529
T3 - Proceedings - IEEE International Conference on Robotics and Automation
SP - 2519
EP - 2524
BT - 2022 IEEE International Conference on Robotics and Automation, ICRA 2022
PB - Institute of Electrical and Electronics Engineers Inc.
Y2 - 23 May 2022 through 27 May 2022
ER -