Semantic state prediction in robotic cloth manipulation

Conference Article


Intelligent Systems Conference (IntelliSys)





Doc link


Download the digital copy of the doc pdf document


State estimation of deformable objects such as textiles is notorious difficult due to its extreme high dimensionality and complexity. Lack of data and benchmarks is another challenge impeding progress in robotic cloth manipulation. In this paper, we make a first attempt to solve the problem of semantic state estimation through RGB-D data only in an end-to-end manner with the help of deep neural networks. Since neural networks require large amounts of labeled data, we introduce a novel Mujoco simulator to generate a large-scale fully annotated robotic textile manipulation dataset including bimanual actions. Finally, we provide a set of baseline deep neural networks and benchmark them on the problem of semantic state prediction on our proposed dataset.


image classification.

Author keywords

Semantic States, Neural Networks, Synthetic Dataset, Mujoco Simulator

Scientific reference

G. Tzelepis, J. Borràs, A. Eren Erdal and G. Alenyà. Semantic state prediction in robotic cloth manipulation, 2023 Intelligent Systems Conference, 2023, Amsterdam, pp. 205-219, Springer Lecture Notes in Networks and Systems.