Publication

Grasp-oriented fine-grained cloth segmentation without real supervision

Conference Article

Conference

International Conference on Machine Vision and Applications (ICMVA)

Edition

2023 6th

Pages

147-153

Doc link

https://dl.acm.org/doi/10.1145/3589572.3589594

File

Download the digital copy of the doc pdf document

Authors

Abstract

Automatically detecting graspable regions from a single depth image is a key ingredient in cloth manipulation. The large variability of cloth deformations has motivated most of the current approaches to focus on identifying specific grasping points rather than semantic parts, as the appearance and depth variations of local regions are smaller and easier to model than the larger ones. However, tasks like cloth folding or assisted dressing require recognizing larger segments, such as semantic edges that carry more information than points. We thus first tackle the problem of fine-grained region detection in deformed clothes using only a depth image. We implement an approach for T-shirts, and define up to 6 semantic regions of varying extent, including edges on the neckline, sleeve cuffs, and hem, plus top and bottom grasping points. We introduce a U-Net based network to segment and label these parts. Our second contribution is concerned with the level of supervision required to train the proposed network. While most approaches learn to detect grasping points by combining real and synthetic annotations, in this work we propose a multilayered Domain Adaptation strategy that does not use any real annotations. We thoroughly evaluate our approach on real depth images of a T-shirt annotated with fine-grained labels, and show that training our network only with synthetic labels and our proposed DA approach yields results competitive with real data supervision.

Categories

computer vision, manipulators, robot vision.

Author keywords

perception for grasping and manipulation, segmentation and categorization, domain adaptation

Scientific reference

R. Ren, M. Gurnani Rajesh, J. Sanchez, A. López, F. Zhang, Y. Tian, G. Alenyà, A. Agudo, Y. Demiris, K. Mikolajczyk and F. Moreno-Noguer. Grasp-oriented fine-grained cloth segmentation without real supervision, 2023 6th International Conference on Machine Vision and Applications, 2023, Singapore, pp. 147-153.