Irene Garcia-Camacho, Júlia Borràs and Guillem Alenyà
Introduction
Cloth manipulation is very relevant for domestic robotic tasks, but it presents many challenges due to the complexity of representing, recognizing and predicting the behaviour of cloth under manipulation. In this work, we propose a generic, compact and simplified representation of the states of cloth manipulation that allows for representing tasks as sequences of states and transitions semantically. We also define a Cloth Manipulation Graph that encodes all the strategies to accomplish a task. Our novel representation is used to encode two different cloth manipulation tasks, learned from an experiment with human subjects with video data. We show how our simplified representation allows to obtain a map of meaningful steps. Finally, we also show how our representation can serve to describe cloth manipulation tasks in PDDL language, enabling high-level planning and discuss on the existing skills that could enable the sensory motor grounding and the low-level execution of the plan.
Video
CloM Graphs
Fold a napking in 3 folds (simplified graph)
Fold a napking in 3 folds (full graph)
In red are the edges that appeared already in the reduced CloM Graph above.
Unfold and put a tablecloth (min. multiplicity 3)
Unfold and put a tablecloth (min. multiplicity 2)
Unfold and put a tablecloth (full graph)
Simplifications assumed for the graphs above DataSimplifications.pdf.
CloM Graph around "Trace Edge" transition with trajectories
The trajectories are translated to start all from the same locations on the right and left hands, and the elipsoids correspond to the average location of the starting points for the left and the right hands
If you have any question or suggestion, don't hesitate to contact me: jborras[at]iri.upc.edu.
Setup for the human demonstrations
|
|
|---|---|
| Picture of the setup, with a cenital depth camera, the XSens suit and the grippers worn by the subject. | The gripper used by the subjects. |
Frames from all the identified states
Here you can find zip files containing all the images named as the state name, numerated from 1 to n according to the number of times that state has been found in our experiments for the two tasks of Folding a napking and Unfolding and putting a tablecloth. The zip files contain also a txt file with the list of files associated to each state tag.
Initial frames of all states
The labeled videos generate segments of video until the state changes. Here we show the intial frame of all the segments. Several frames could be used if more data is needed.
Click here to download the zip file: zip
Middle frames of all states
The labeled videos generate segments until the state changes. Here we show the middle frame of all the segments.
Click here to download the zip file: zip
Planning domain model generator
The repository to generate the LISP-like domain and problem files in PDDL is available at Github with instructions on how to use it and examples of the two cloth manipulation tasks: folding a napkin in 3 folds and setting a tablecloth.