Research line

Perception and Manipulation Image

The research of PERCEPTION AND MANIPULATION group focuses on enhancing the perception, learning, and planning capabilities of robots to achieve higher degrees of autonomy and user-friendliness during everyday manipulation tasks. Some topics addressed are the geometric interpretation of perceptual information, construction of 3D object models, action selection and planning, reinforcement learning, and teaching by demonstration.

Head of line: Carme Torras Genís

Head of line

Tech. transfer

Our activity finds applications in several fields through collaboration with our technological partners

Research projects

We carry out projects from national and international research programmes.
→ More about our research projects

<< Back to Perception and Manipulation main page

Learning by demonstration

We devise methods to learn object-action relations to accomplish tasks at different levels of abstraction, where object models are generated from visual and depth information, and actions, involving manipulation skills, are learned from demonstrations provided by a human using multimodal algorithms that combine vision and haptics.

→ more info

Research area 1 of Perception and Manipulation

Planning for perception and manipulation

We are interested in view planning for object modeling, as well as manipulation planning, with special interest in deformable objects. High-level task formulations are integrated with low-level geometry-based methods and simplified physical models, together with an on-line sensory-based treatment of uncertainty, so as to come up with specific sequences of motion commands.

→ more info

Research area 2 of Perception and Manipulation

Perception of rigid and non-rigid objects

Our objective is to investigate computer vision algorithms for interpreting and understanding scenes from images, with applications in robotics and medical imaging. In particular, our activities are concentrated on retrieving rigid and non-rigid shape, motion and camera pose from single images and video sequences.

→ more info

Research area 3 of Perception and Manipulation

These are the latest research projects of the Perception and Manipulation research line:

These are the most recent publications (2014 - 2013) of the Perception and Manipulation

  • W. Kazmi, S. Foix, G. Alenyà and H.J. Andersen. Indoor and outdoor depth imaging of leaves with time-of-flight and stereo vision sensors: Analysis and comparison. ISPRS Journal of Photogrammetry and Remote Sensing, 88: 128-146, 2014.

    Open/Close abstract Abstract Info Info pdf PDF
  • E. Simo-Serra and A. Perez-Gracia. Kinematic synthesis using tree topologies. Mechanism and Machine Theory, 72: 94-113, 2014.

    Open/Close abstract Abstract Info Info pdf PDF
  • F. Husain, A. Colomé, B. Dellen, G. Alenyà and C. Torras. Realtime tracking and grasping of a moving object from range video, 2014 IEEE International Conference on Robotics and Automation, 2014, Hong Kong, pp. 2617-2622, to appear.

    Open/Close abstract Abstract Info Info pdf PDF
  • D. Martínez, G. Alenyà, P. Jiménez, C. Torras, J. Rossmann, N. Wantia, A. Eren Erdal, S. Haller and J. Piater. Active learning of manipulation sequences, 2014 IEEE International Conference on Robotics and Automation, 2014, Hong Kong, pp. 5671-5678, to appear.

    Open/Close abstract Abstract Info Info pdf PDF
  • G. Alenyà, J.L. Rivero, A. Rull, P. Grosch and S. Hernández. The HumanoidLab: Involving students in a research centre through an educational initiative, 6th International Conference on Computer Supported Education, 2014, Barcelona, pp. 213-220, Scitepress.

    Open/Close abstract Abstract Info Info pdf PDF
  • Y. Isleyici and G. Alenyà. Teaching grasping points using natural movements. Technical Report IRI-TR-14-01, Institut de Robòtica i Informàtica Industrial, CSIC-UPC, 2014.

    Open/Close abstract Abstract Info Info pdf PDF
  • E. Trulls Fortuny, S. Tsogkas, I. Kokkinos, A. Sanfeliu and F. Moreno-Noguer. Segmentation-aware deformable part models, 2014 IEEE Computer Society Conference on Computer Vision and Pattern Recognition, 2014, Columbus, Ohio (USA), to appear.

    Open/Close abstract Abstract Info Info pdf PDF
  • A. Colomé and C. Torras. Closed-loop inverse kinematics for redundant robots: Comparative assessment and two enhancements. IEEE/ASME Transactions on Mechatronics, 2014, to appear.

    Open/Close abstract Abstract Info Info pdf PDF
  • G. Alenyà, S. Foix and C. Torras. ToF cameras for active vision in robotics. Sensors and Actuators A: Physical, 218: 10-22, 2014.

    Open/Close abstract Abstract Info Info pdf PDF
  • A. Ramisa, G. Alenyà, F. Moreno-Noguer and C. Torras. Learning RGB-D descriptors of garment parts for informed robot grasping. Engineering Applications of Artificial Intelligence, 35: 246-258, 2014.

    Open/Close abstract Abstract Info Info pdf PDF
  • F. Husain, B. Dellen and C. Torras. Consistent depth video segmentation using adaptive surface models. IEEE Transactions on Cybernetics, 2014, to appear.

    Open/Close abstract Abstract Info Info pdf PDF
  • G. Alenyà, S. Foix and C. Torras. Using ToF and RGBD cameras for 3D robot perception and manipulation in human environments. Intelligent Service Robotics, 2014, to appear.

    Open/Close abstract Abstract Info Info pdf PDF
  • D. Martínez, G. Alenyà and C. Torras. Planning surface cleaning tasks by learning uncertain drag actions outcomes, 2013 ICAPS Workshop on Planning and Robotics, 2013, Rome, pp. 106-111.

    Open/Close abstract Abstract Info Info pdf PDF
  • A. Garrell Zulueta, M. Villamizar, F. Moreno-Noguer and A. Sanfeliu. Proactive behavior of an autonomous mobile robot for human-assisted learning, 22nd IEEE International Symposium on Robot and Human Interactive Communication, 2013, Gyeongju, Korea, pp. 107-113.

    Open/Close abstract Abstract Info Info pdf PDF
  • J. Serra and G. Alenyà. Robot de telepresencia: imatge i so. Technical Report IRI-TR-13-02, Institut de Robòtica i Informàtica Industrial, CSIC-UPC, 2013.

    Open/Close abstract Abstract Info Info pdf PDF
  • J. Díaz and C. Torras. Turing's algorithmic lens: From computability to complexity theory. Arbor - Ciencia, Pensamiento y Cultura, 189(764), 2013.

    Open/Close abstract Abstract Info Info pdf PDF
  • B. Dellen and C. Torras. Local stimulus disambiguation with global motion filters predicts adaptive surround modulation. Neural Networks, 46: 32-39, 2013.

    Open/Close abstract Abstract Info Info pdf PDF
  • M.A. Pinheiro, R. Sznitman, E. Serradell, J. Kybic, F. Moreno-Noguer and P. Fua. Active testing search for point cloud matching, 23rd International Conference on Information Processing in Medical Imaging, 2013, Asilimar, USA, in Information Processing in Medical Imaging, Vol 7917 of Lecture Notes in Computer Science, pp. 572-583, Springer-Verlag.

    Open/Close abstract Abstract Info Info pdf PDF
  • G. Alenyà, B. Dellen, S. Foix and C. Torras. Robotized plant probing: Leaf segmentation utilizing time-of-flight data . IEEE Robotics and Automation Magazine, 20(3): 50-59, 2013.

    Open/Close abstract Abstract Info Info pdf PDF
  • A. Gonzalez and G. Alenyà. The MoveIt motion planning framework Configuration for the IRI WAM robot. Technical Report IRI-TR-13-03, Institut de Robòtica i Informàtica Industrial, CSIC-UPC, 2013.

    Open/Close abstract Abstract Info Info pdf PDF
  • A. Penate-Sanchez, E. Serradell, J. Andrade-Cetto and F. Moreno-Noguer. Simultaneous pose, focal length and 2D-to-3D correspondences from noisy observations, 2013 British Machine Vision Conference, 2013, Bristol, UK, pp. 82.1-82.11.

    Open/Close abstract Abstract Info Info pdf PDF
  • P. Jiménez. Survey on assembly sequencing: A combinatorial and geometrical perspective. Journal of Intelligent Manufacturing, 24(2): 235-250, 2013.

    Open/Close abstract Abstract Info Info pdf PDF
  • A. Ramisa and C. Torras. Large-scale image classification using ensembles of nested dichotomies, 16th Catalan Conference on Artificial Intelligence, 2013, Vic, Spain, pp. 87-90, IOS Press.

    Open/Close abstract Abstract Info Info pdf PDF
  • J.J. McAuley, A. Ramisa and T.S. Caetano. Optimization of robust loss functions for weakly-labeled image taxonomies. International Journal of Computer Vision, 104(3): 343-361, 2013.

    Open/Close abstract Abstract Info Info pdf PDF
  • A. Ramisa, G. Alenyà, F. Moreno-Noguer and C. Torras. FINDDD: A fast 3D descriptor to characterize textiles for robot manipulation, 2013 IEEE/RSJ International Conference on Intelligent Robots and Systems, 2013, Tokyo, pp. 824-830.

    Open/Close abstract Abstract Info Info pdf PDF
  • B. Dellen, F. Husain and C. Torras. Joint segmentation and tracking of object surfaces in depth movies along human/robot manipulations, 8th International Conference on Computer Vision Theory and Applications, 2013, Barcelona, pp. 244-251.

    Open/Close abstract Abstract Info Info pdf PDF
  • B. Dellen and I.A. Rojas. Volume measurement with a consumer depth camera based on structured infrared light, 16th Catalan Conference on Artificial Intelligence, poster session, 2013, Vic, Spain, pp. 1-10, ACIA.

    Open/Close abstract Abstract Info Info pdf PDF
  • A. Colomé, D. Pardo, G. Alenyà and C. Torras. External force estimation during compliant robot manipulation, 2013 IEEE International Conference on Robotics and Automation, 2013, Karlsruhe, pp. 3535-3540.

    Open/Close abstract Abstract Info Info pdf PDF
  • E. Trulls Fortuny, I. Kokkinos, A. Sanfeliu and F. Moreno-Noguer. Dense segmentation-aware descriptors, 2013 IEEE Computer Society Conference on Computer Vision and Pattern Recognition, 2013, Portland, USA, pp. 2890-2897.

    Open/Close abstract Abstract Info Info pdf PDF
  • L. Rozo, P. Jiménez and C. Torras. A robot learning from demonstration framework to perform force-based manipulation tasks. Intelligent Service Robotics, 6(1): 33-51, 2013.

    Open/Close abstract Abstract Info Info pdf PDF
  • L. Rozo, S. Calinon, D. Caldwell, P. Jiménez and C. Torras. Learning collaborative impedance-based robot behaviors, Twenty-Seventh AAAI Conference on Artificial Intelligence, 2013, Bellevue, pp. 1422-1428.

    Open/Close abstract Abstract Info Info pdf PDF
  • S. Reich, A. Abramov, J. Papon, F. Wörgötter and B. Dellen. A novel real-time edge-preserving smoothing filter, 8th International Conference on Computer Vision Theory and Applications, 2013, Barcelona, pp. 1-11.

    Open/Close abstract Abstract Info Info pdf PDF
  • L. Rozo, P. Jiménez and C. Torras. Force-based robot learning of pouring skills using parametric hidden Markov models, 9th International Workshop on Robot Motion and Control, 2013, Wasowo, Poland, pp. 227-232.

    Open/Close abstract Abstract Info Info pdf PDF
  • F. Moreno-Noguer and P. Fua. Stochastic exploration of ambiguities for nonrigid shape recovery. IEEE Transactions on Pattern Analysis and Machine Intelligence, 35(2): 463-475, 2013.

    Open/Close abstract Abstract Info Info pdf PDF
  • E. Todt and C. Torras. Outdoor view recognition based on landmark grouping and logistic regression. International Journal of Pattern Recognition and Artificial Intelligence , 27(3): 1-21, 2013.

    Open/Close abstract Abstract Info Info pdf PDF
  • A. Penate-Sanchez, J. Andrade-Cetto and F. Moreno-Noguer. Exhaustive linearization for robust camera pose and focal length estimation. IEEE Transactions on Pattern Analysis and Machine Intelligence, 35(10): 2387-2400, 2013.

    Open/Close abstract Abstract Info Info pdf PDF
  • F. Ballesté and C. Torras. Effects of human-machine integration on the construction of identity. In Handbook of Research on Technoself: Identity in a Technological Society, 574-591. IGI Global, 2013.

    Open/Close abstract Abstract Info Info pdf PDF
  • A. Colomé, G. Alenyà and C. Torras. Handling high parameter dimensionality in reinforcement learning with dynamic motor primitives, 2013 ICRA Workshop on Novel Methods for Learning and Optimization of Control Policies and Trajectories for Robotics, 2013, Karlsruhe, Germany.

    Open/Close abstract Abstract Info Info pdf PDF
  • C. Torras. From the Turing test to science fiction: The challenges of social robotics, 16th Catalan Conference on Artificial Intelligence, 2013, Vic, Spain, pp. 5-7, IOS Press.

    Open/Close abstract Abstract Info Info pdf PDF
  • A. Colomé and C. Torras. Positioning two redundant arms for cooperative manipulation of objects, 6th International Workshop on Computational Kinematics, 2013, Barcelona, in Computational Kinematics, Vol 15 of Mechanisms and Machine Science, pp. 121-129, 2014, Springer.

    Open/Close abstract Abstract Info Info pdf PDF
  • E. Simo-Serra, A. Quattoni, C. Torras and F. Moreno-Noguer. A joint model for 2D and 3D pose estimation from a single image, 2013 IEEE Computer Society Conference on Computer Vision and Pattern Recognition, 2013, Portland, USA, pp. 3634--3641.

    Open/Close abstract Abstract Info Info pdf PDF

Perception and Manipulation Laboratory

The Perception and Manipulation Laboratory is equipped with 2 workcells: one with 2 standard manipulators and an XY positioner, and the other with 2 WAM arms in a reconfigurable arrangement. Additionally, researchers can find a 3 fingered hand, a Delta haptic interface, force sensors, several conventional cameras, and high speed, high resolution, and 3D cameras. Laboratory service offers quick experimental setup, several standardized software tools, and expertise in robot control and perception algorithms. It also hosts the Humanoid Lab initiative, with 15 small humanoid robots.

Perception and Manipulation Laboratory
Group photo

Researchers

PhD Students

Master Students

PFC Students