Publication
Recognizing object surface material from impact sounds for robot manipulation
Conference Article
Conference
IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS)
Edition
2022
Pages
9280-9287
Doc link
https://doi.org/10.1109/IROS47612.2022.9981578
File
Authors
-
Dimiccoli, Mariella
-
Patni, Shubhan
-
Hoffmann, Matej
-
Moreno Noguer, Francesc
Projects associated
Abstract
We investigated the use of impact sounds generated during exploratory behaviors in a robotic manipulation setup as cues for predicting object surface material and for recognizing individual objects. We collected and make available the YCB-impact sounds dataset which includes over 3,000 impact sounds for the YCB set of everyday objects lying on a table. Impact sounds were generated in three modes: (i) human holding a gripper and hitting, scratching, or dropping the object; (ii) gripper attached to a teleoperated robot hitting the object from the top; (iii) autonomously operated robot
hitting the objects from the side with two different speeds.
A convolutional neural network is trained from scratch to recognize the object material (steel, aluminium, hard plastic,
soft plastic, other plastic, ceramic, wood, paper/cardboard, foam, glass, rubber) from a single impact sound. On the manually collected dataset with more variability in the speed of the action, nearly 60% accuracy for the test set (not presented objects) was achieved. On a robot setup and a stereotypical poking action from top, accuracy of 85% was achieved. This performance drops to 79% if multiple exploratory actions are combined. Individual objects from the set of 75 objects can be recognized with a 79% accuracy. This work demonstrates promising results regarding the possibility of using impact sound for recognition in tasks like single-stream recycling where objects have to be sorted based on their material composition.
Categories
industrial robots, intelligent robots, manipulators.
Author keywords
object properties, material predictino, audio perception
Scientific reference
M. Dimiccoli, S. Patni, M. Hoffmann and F. Moreno-Noguer. Recognizing object surface material from impact sounds for robot manipulation, 2022 IEEE/RSJ International Conference on Intelligent Robots and Systems, 2022, Kyoto, pp. 9280-9287.
Follow us!