Research line

Perception and Manipulation

Perception and Manipulation Image

The research of PERCEPTION AND MANIPULATION group focuses on enhancing the perception, learning, and planning capabilities of robots to achieve higher degrees of autonomy and user-friendliness during everyday manipulation tasks. Some topics addressed are the geometric interpretation of perceptual information, construction of 3D object models, action selection and planning, reinforcement learning, and teaching by demonstration.
❯ See our presentation video ❯

Head of line: Carme Torras Genís

Head of line

Tech. transfer

Our activity finds applications in several fields through collaboration with our technological partners

Research projects

We carry out projects from national and international research programmes.
→ More about our research projects

<< Back to Perception and Manipulation main page

Learning by demonstration

We devise methods to learn object-action relations to accomplish tasks at different levels of abstraction, where object models are generated from visual and depth information, and actions, involving manipulation skills, are learned from demonstrations provided by a human using multimodal algorithms that combine vision and haptics.

→ more info

Research area 1 of Perception and Manipulation

Planning for perception and manipulation

We are interested in view planning for object modeling, as well as manipulation planning, with special interest in deformable objects. High-level task formulations are integrated with low-level geometry-based methods and simplified physical models, together with an on-line sensory-based treatment of uncertainty, so as to come up with specific sequences of motion commands.

→ more info

Research area 2 of Perception and Manipulation

Perception of rigid and non-rigid objects

Our objective is to investigate computer vision algorithms for interpreting and understanding scenes from images, with applications in robotics and medical imaging. In particular, our activities are concentrated on retrieving rigid and non-rigid shape, motion and camera pose from single images and video sequences.

→ more info

Research area 3 of Perception and Manipulation

These are the latest research projects of the Perception and Manipulation research line:

  • ---: ---

    Technology Transfer Contract

    Start Date: 24/10/2003

These are the most recent publications (2026 - 2025) of the Perception and Manipulation

  • F. Coltraro, J. Amorós, M. Alberich-Carramiñana and C. Torras. Topological reconstruction of sampled surfaces via Morse theory. Journal of Computational and Applied Mathematics, 473: 116872, 2026.

    Open/Close abstract Abstract Info Info pdf PDF

  • A. Aryania, S. Chockalingam, H.K. Rødsethol and G. Alenyà. Impact of design transparency on trust and data sharing during human-robot interactions in public places. ACM Transactions on Human-Robot Interaction, 2026.

    Open/Close abstract Abstract Info Info pdf PDF

  • D.F. Ordoñez, G. Turrisi, V. Kostic, M. Martin, A. Agudo, F. Moreno-Noguer, M. Pontil, C. Semini and C. Mastalli. Morphological symmetries in robotics. International Journal of Robotics Research , 44(10): 1743-1766, 2025.

    Open/Close abstract Abstract Info Info pdf PDF

  • E. Caldarelli, A. Chatalic, A. Colomé, C. Molinari, C. Ocampo-Martínez, C. Torras and L. Rosasco. Linear quadratic control of nonlinear systems with Koopman operator learning and the Nyström method. Automatica, 177: 112302, 2025.

    Open/Close abstract Abstract Info Info pdf PDF

  • R. Kermenov, S. Foix, J. Borràs, V. Castorani, S. Longhi and A. Bonci. Automating the hand layup process: On the removal of protective films with collaborative robots. Robotics and Computer-Integrated Manufacturing, 93: 102899, 2025, to appear.

    Open/Close abstract Abstract Info Info pdf PDF

  • O. Barbany, A. Colomé and C. Torras. BiFold: Bimanual cloth folding with language guidance, 2025 IEEE International Conference on Robotics and Automation, 2025, Atlanta (GA, USA), pp. 5652-5659.

    Open/Close abstract Abstract Info Info pdf PDF

  • A. Longhini, Y. Wang, I. Garcia-Camacho, D. Blanco-Mulero, M. Moletta, M.C. Welle, G. Alenyà, H. Yin, Z. Erickson, D. Held, J. Borràs and D. Kragic. Unfolding the literature: a review of robotic cloth manipulation. Annual Review of Control, Robotics, and Autonomous Systems, 8: 295-322, 2025.

    Open/Close abstract Abstract Info Info pdf PDF

  • O. Barbany, A. Colomé and C. Torras. Beyond static perception: Integrating temporal context into VLMs for cloth folding, 2025 ICRA Workshop on Representing and Manipulating Deformable Objects, 2025, , pp. 1-4.

    Open/Close abstract Abstract Info Info pdf PDF

  • S. Singhal, R. Pérez, A. Espersen and A. Agudo. Dual-space augmented intrinsic-LoRA for wind turbine segmentation, 2025 IEEE International Conference on Acoustics, Speech and Signal Processing, 2025, Hyderabad, India, pp. 1-5.

    Open/Close abstract Abstract Info Info pdf PDF

  • P. Caselles, E. Ramon, J. Garcia, G. Triginer and F. Moreno-Noguer. Implicit shape and appearance priors for few-shot full head reconstruction. IEEE Transactions on Pattern Analysis and Machine Intelligence, 47(5): 3691-3705, 2025.

    Open/Close abstract Abstract Info Info pdf PDF

  • M. Alberich-Carramiñana, J. Guàrdia, E. Nart, A. Poteaux, J. Roé and M. Weimann. Polynomial factorization over Henselian fields. Foundations of computational mathematics, 2025.

    Open/Close abstract Abstract Info Info pdf PDF

  • E.B. Bueno Benito and M. Dimiccoli. CLOT: Closed Loop Optimal Transport for unsupervised action segmentation, 2025 International Conference on Computer Vision, 2025, Honolulu, Hawai'i, pp. 10719-10729.

    Open/Close abstract Abstract Info Info pdf PDF

  • N. Vallès and J. Pareto. Artificial intelligence as a mode of ordering. Automated-decision making in primary care. Information, Communication & Society, 2025, to appear.

    Open/Close abstract Abstract Info Info pdf PDF

  • A. Martínez and S. Foix. Continuous object orientation estimation of cotton crops using deep convolutional neural networks, 2025 Catalan Conference on Artificial Intelligence, poster session, 2025, Valls, Spain, pp. 270-274, IOS Press.

    Open/Close abstract Abstract Info Info pdf PDF

  • F. Coltraro, J. Borràs, M. Alberich-Carramiñana and C. Torras. Tracking cloth deformation: a novel dataset for closing the sim-to-real gap for robotic cloth manipulation learning. International Journal of Robotics Research , 44(9): 1431-1442, 2025.

    Open/Close abstract Abstract Info Info pdf PDF

  • G. González, S. Foix and G. Alenyà. An improved YOLO11 model for accurate cotton growth-stage detection, 2025 Catalan Conference on Artificial Intelligence, poster session, 2025, Valls, Spain, pp. 266-269, IOS Press.

    Open/Close abstract Abstract Info Info pdf PDF

  • G. Capellera, A. Rubio, L. Ferraz and A. Agudo. Unified uncertainty-aware diffusion for multi-agent trajectory modeling, 2025 IEEE/CVF Conference on Computer Vision and Pattern Recognition, 2025, Nashville, TN (USA), pp. 22476-22486.

    Open/Close abstract Abstract Info Info pdf PDF

  • D. Blanco-Mulero, J. Borràs and C. Torras. Evaluating the pre-dressing step: Unfolding medical garments via imitation learning, 2025 IEEE/RSJ International Conference on Intelligent Robots and Systems, 2025, Hanzghou, China, pp. 17791-17796.

    Open/Close abstract Abstract Info Info pdf PDF

  • M. Peral, G. Capellera, A. Rubio, L. Ferraz, F. Moreno-Noguer and A. Agudo. Temporally accurate events detection through ball possessor recognition in soccer, 2025 International Conference on Computer Vision Theory and Applications, 2025, Porto, Portugal, pp. 221-231.

    Open/Close abstract Abstract Info Info pdf PDF

  • R. Valero, C.O. Diaz and J. Sanchez. Real-time game highlight detection for data-driven league of legends coaching, 2025 Workshop on Data Engineering and Analytics, 2025, Ben Guerir, Morocco, pp. 211-223.

    Open/Close abstract Abstract Info Info pdf PDF

  • A. Aryania, R. Huertas, S. Forgas, C. Angulo and G. Alenyà. Effects of privacy warning on the intention to disclose personal information during interaction with a robot in public spaces. International Journal of Social Robotics, 17: 1055–1073, 2025.

    Open/Close abstract Abstract Info Info pdf PDF

  • A. Andriella, C. Torras and G. Alenyà. Personalising human-robot interactions in social contexts. Volume 159 of Springer Tracts in Advanced Robotics. Springer, 2025.

    Open/Close abstract Abstract Info Info

  • A. Favata, R. Gallart, L. van Noort, J. Exposito, J. Medina-Cantillo, C. Torras, D. Natera-de Benito, J.M. Font and R. Pàmies-Vilà. Imu-based kinematic analysis to enhance upper limb motor function assessment in neuromuscular diseases. Journal of NeuroEngineering and Rehabilitation, 22(63), 2025.

    Open/Close abstract Abstract Info Info pdf PDF

  • M. Gutiérrez and A. Agudo. SoccerNet-v3D: leveraging sports broadcast replays for 3D scene understanding, 2025 IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops (CVPRW). Workshop on Computer Vision in Sports, 2025, Nashville (TN, USA), pp. 5968-5977.

    Open/Close abstract Abstract Info Info pdf PDF

  • M. Méndez, A. Fornés and A. Agudo. Recovering and classifying upper limb impairment trajectories after stroke, 2025 IEEE International Conference on Image Processing, 2025, Anchorage (AK, USA), pp. 1630-1635.

    Open/Close abstract Abstract Info Info pdf PDF

  • J. Morales, B. Gebregziabher, A. Cabañeros and J. Sanchez. VQA-driven event maps for assistive navigation for people with low vision in urban environments, 2025 IEEE International Conference on Robotics and Automation, 2025, Atlanta (GA, USA), pp. 7, to appear.

    Open/Close abstract Abstract Info Info pdf PDF

  • P. Jiménez and C. Torras. Robots. Una immersió rápida. Volume 26 of Una immersió rápida. Tibidabo Edicions, SA, 2025.

    Open/Close abstract Abstract Info Info

  • F. Coltraro, J. Amorós, C. Torras and M. Alberich-Carramiñana. A practical aerodynamic model for dynamic textile manipulation in robotics. Mechanism and Machine Theory, 209(105993), 2025.

    Open/Close abstract Abstract Info Info pdf PDF

  • L. Fuster, M. Dalmasso, A. Aubach, S. Izquierdo, A. Sanfeliu and A. Garrell Zulueta. Negotiation of assignation plans in human-robot team task scheduling, 34th IEEE International Symposium on Robot and Human Interactive Communication, 2025, Eindhoven, Netherlands, pp. 1222-1228.

    Open/Close abstract Abstract Info Info pdf PDF

Perception and Manipulation Laboratory

The Perception and Manipulation Laboratory is the main facility devoted to experimentation on the topics of the research line. It occupies 142 m² in the second floor of the Faculty, and a large part hosts a life-scale mock-up of a (quite) small apartment (35 m²). Two PAL one-armed mobile TIAGo robots dwell within the apartment, but are occasionally taken out for experimentation elsewhere. There is also a manipulation area equipped with two WAM robot arms and a KINOVA manipulator, and workplaces are distributed along the perimeter. Next to the laboratory, with a direct view through a window, there is the Scientific and Technical Lab support office.

The Laboratory is further equipped with commercial and self-developed grippers, sensing devices, and augmented reality appliances. Laboratory service offers quick experimental setup, several standardized software tools, and expertise in robot control and perception algorithms. It also hosted the Humanoids Lab Initiative in the past, and we still keep 15 small humanoid robots for educational or promotional purposes.

Perception and Manipulation Laboratory
Group photo

Researchers

PhD Students

Master Students

TFG Students

Support Staff