Project Description

Project PID2020-118649RB-I00 funded by MCIN/ AEI /10.13039/501100011033

CHLOE-GRAPH proposes a novel robotic manipulation approach for cloth manipulation that is expected to be general enough to be applied to different assistive tasks. While rigid object manipulation has achieved a decent maturity, robots are yet to make an impact in the handling of deformable objects. Cloth-like objects are extremely challenging due to the complexity of understanding their deformation often with severe self-occlusions, making learning from real data very difficult. This has led to a tendency to develop techniques focused on specific tasks, resulting in a lack of general approaches that provide solutions that are transferable to different object types and tasks. Robots in assistive environments, able to manipulate cloth-like objects, would be a major step, able to provide new business opportunities.
In CHLOE-GRAPH we argue that the solution must embrace the different robotic layers, perception, decision making, and execution. We propose to develop a novel CHLOE (ClotH-Like ObjEcts) scene state representation (SSR) carefully designed to simplify its recognition, but including detailed information on the parts that have to be manipulated to execute the next action and also semantic tags designed for high-level planning.


The 4 objectives of CHLOE-GRAPH are

(O1) - To define the CHLOE-SSR by selecting semantic tags and parameters depending on the requirements from perception, execution, and high-level planning of manipulation tasks with textiles. The representation will have a central role: it will help to determine what has to be perceived, will be used to bootstrap planning operators, and will define the desired abilities of the manipulation actions.

(O2) To develop a CHLOE-SSR recognition method based on deep learning from real data labelled with the novel representation tags. We plan to combine computational topological methods with deep learning architectures to identify only the relevant features of the 2D surface in a 3D world, such as the environment contacts, the grasp location, the deformation category, and other manipulation affordance related information. We propose that only the relevant parts for manipulation need to be understood, not the full deformation state.

(O3) To implement a set of low-level primitives with strategies to accomplish transitions from state to state using the information in CHLOE-SSR. Depending on the complexity of each transition, we will use learning from demonstration techniques, end-to-end deep learning sim-to-real methods with dynamics extending from previous results, or model predictive control, implementing the parameters of the SSR within the motion characterization to allow for trajectory adaptability to the environment.

(O4) To develop a high-level decision-making framework to decide robot actions among the alternative plans, taking into account the non-complete observability of the state and the uncertainty on the action execution. We will contribute to three challenges: the representation, the complexity related to the high-low level gap, and the use of experiences to refine the learned transition model.
We will also propose novel measures to benchmark cloth manipulation tasks related to the segmentation steamed from the new representation, and demonstrate the validity of our approach in two robotic demonstrators.