Irene Garcia-Camacho, Júlia Borràs and Guillem Alenyà

Abstract - Benchmarking robot manipulations is complex due to the difficulty in reproducing and comparing results across different embodiments and scenarios. Cloth manipulation presents additional challenges due to the complex object configuration space, making it difficult to define initial poses, to find standard objects sets or assessing the quality of a task. Traditional cloth manipulation papers do not have well defined metrics to evaluate the success of a task or the quality of the result, and are tailored to each evaluation. In this paper we propose to evaluate cloth manipulation using action graphs that model a task as a graph of transitions between different scene states. This results into an action segmentation that allows us to evaluate each segment, its influence in the next segment and relate it to task as a whole. Using a common bi-manual cloth manipulation task such as placing a cloth flat on a table, we propose a benchmark with simple evaluation metrics that explore the influence of grasp location and trajectory into the quality of the task. Our results show that grasp location doesn’t need to be precise on corners, that quality measures focused on evaluating different cloth parts can enlighten issues to solve and that success definition of a segment has to consider its influence on the ability to perform successfully the next segment of action.

The benchmark

Setup description

The objects used are listed in the Table 1 and includes three rectangular napkins and towels of different sizes and yarns.

Initial grasping points

The initial points from where to grasp the objects are defined as the vertices of a grid of 9 points from the left corner of the object (LC) and the 25% of l (length of the cloth edge between the two robotic arms), in addition to two more points at a distance of 37.5% of l.

Evaluation Metrics

The repository to evaluate the placed cloth on top of the table is avaliable on Github with instructions on how to use it and examples of real results of the implemented baseline to test the packages.

Video

Citation

I. Garcia-Camacho, J. Borràs and G. Alenyà. Benchmarking cloth manipulation using action graphs: an example in placing flat, 2021 IROS Workshop on Benchmarking of robotic grasping and manipulation: protocols, metrics and data analysis, 2021, Online, pp. 1-3