Research line

Perception and Manipulation Image

The research of PERCEPTION AND MANIPULATION group focuses on enhancing the perception, learning, and planning capabilities of robots to achieve higher degrees of autonomy and user-friendliness during everyday manipulation tasks. Some topics addressed are the geometric interpretation of perceptual information, construction of 3D object models, action selection and planning, reinforcement learning, and teaching by demonstration.

Head of line: Carme Torras Genís

Head of line

Tech. transfer

Our activity finds applications in several fields through collaboration with our technological partners

Research projects

We carry out projects from national and international research programmes.
→ More about our research projects

<< Back to Perception and Manipulation main page

Learning by demonstration

We devise methods to learn object-action relations to accomplish tasks at different levels of abstraction, where object models are generated from visual and depth information, and actions, involving manipulation skills, are learned from demonstrations provided by a human using multimodal algorithms that combine vision and haptics.

→ more info

Research area 1 of Perception and Manipulation

Planning for perception and manipulation

We are interested in view planning for object modeling, as well as manipulation planning, with special interest in deformable objects. High-level task formulations are integrated with low-level geometry-based methods and simplified physical models, together with an on-line sensory-based treatment of uncertainty, so as to come up with specific sequences of motion commands.

→ more info

Research area 2 of Perception and Manipulation

Perception of rigid and non-rigid objects

Our objective is to investigate computer vision algorithms for interpreting and understanding scenes from images, with applications in robotics and medical imaging. In particular, our activities are concentrated on retrieving rigid and non-rigid shape, motion and camera pose from single images and video sequences.

→ more info

Research area 3 of Perception and Manipulation

These are the latest research projects of the Perception and Manipulation research line:

These are the most recent publications (2015 - 2014) of the Perception and Manipulation

  • C. Angulo, S. Pfeiffer, R. Tellez and G. Alenyà. Evaluating the use of robots to enlarge AAL services. Journal of Ambient Intelligence and Smart Environments, 7(3): 301-313, 2015.

    Open/Close abstract Abstract Info Info pdf PDF
  • A. Agostini, C. Torras and F. Wörgötter. Efficient interactive decision-making framework for robotic applications. Artificial Intelligence, 2015, to appear.

    Open/Close abstract Abstract Info Info pdf PDF
  • D. Marquès and G. Alenyà. Planning trajectories for the Esautomatix workcell. Technical Report IRI-TR-15-06, Institut de Robòtica i Informàtica Industrial, CSIC-UPC, 2015.

    Open/Close abstract Abstract Info Info pdf PDF
  • A. Colomé and C. Torras. Closed-loop inverse kinematics for redundant robots: Comparative assessment and two enhancements. IEEE/ASME Transactions on Mechatronics, 20(2): 944-955, 2015.

    Open/Close abstract Abstract Info Info pdf PDF
  • F. Husain, B. Dellen and C. Torras. Consistent depth video segmentation using adaptive surface models. IEEE Transactions on Cybernetics, 45(2): 266-278, 2015.

    Open/Close abstract Abstract Info Info pdf PDF
  • E. Serradell, M.A. Pinheiro, R. Sznitman, J. Kybic, F. Moreno-Noguer and P. Fua. Non-rigid graph registration using active testing search. IEEE Transactions on Pattern Analysis and Machine Intelligence, 37(3): 625-638, 2015.

    Open/Close abstract Abstract Info Info pdf PDF
  • D. Martínez, G. Alenyà and C. Torras. Planning robot manipulation to clean planar surfaces. Engineering Applications of Artificial Intelligence, 39: 23-32, 2015.

    Open/Close abstract Abstract Info Info pdf PDF
  • D. Martínez, G. Alenyà and C. Torras. V-MIN: Efficient reinforcement learning through demonstrations and relaxed reward demands, 2015 AAAI Conference on Artificial Intelligence, 2015, Austin, Texas, to appear.

    Open/Close abstract Abstract Info Info pdf PDF
  • D. Martínez, G. Alenyà and C. Torras. Relational reinforcement learning with guided demonstrations. Artificial Intelligence, 2015, to appear.

    Open/Close abstract Abstract Info Info pdf PDF
  • A. Colomé, A. Planells and C. Torras. A friction-model-based framework for reinforcement learning of robotic tasks in non-rigid environments, 2015 IEEE International Conference on Robotics and Automation, 2015, Seattle, to appear.

    Open/Close abstract Abstract Info Info pdf PDF
  • A. Rubio, M. Villamizar, L. Ferraz, A. Penate-Sanchez, A. Ramisa, E. Simo-Serra, A. Sanfeliu and F. Moreno-Noguer. Efficient monocular pose estimation for complex 3D models, 2015 IEEE International Conference on Robotics and Automation, 2015, Seattle, pp. 1397-1402, to appear.

    Open/Close abstract Abstract Info Info pdf PDF
  • M. Villamizar, A. Garrell Zulueta, A. Sanfeliu and F. Moreno-Noguer. Modeling robot's world with minimal effort, 2015 IEEE International Conference on Robotics and Automation, 2015, Seattle, pp. 4890-4896, to appear.

    Open/Close abstract Abstract Info Info pdf PDF
  • M. Villamizar, A. Garrell Zulueta, A. Sanfeliu and F. Moreno-Noguer. Multimodal object recognition using random clustering trees, 2015 Iberian Conference on Pattern Recognition and Image Analysis, 2015, Santiago de Compostela, Spain, Springer, to appear.

    Open/Close abstract Abstract Info Info pdf PDF
  • G. Alenyà, S. Foix and C. Torras. Using ToF and RGBD cameras for 3D robot perception and manipulation in human environments. Intelligent Service Robotics, 7(4): 211-220, 2014.

    Open/Close abstract Abstract Info Info pdf PDF
  • L. Ferraz, X. Binefa and F. Moreno-Noguer. Leveraging feature uncertainty in the PnP problem, 2014 British Machine Vision Conference, 2014, .

    Open/Close abstract Abstract Info Info pdf PDF
  • A. Colomé and C. Torras. Dimensionality reduction and motion coordination in learning trajectories with dynamic movement primitives, 2014 IEEE/RSJ International Conference on Intelligent Robots and Systems, 2014, Chicago, pp. 1414-1420.

    Open/Close abstract Abstract Info Info pdf PDF
  • E. Simo-Serra, S. Fidler, F. Moreno-Noguer and R. Urtasun. A high performance CRF model for clothes parsing, 12th Asian Conference on Computer Vision, 2014, Singapore, in Computer Vision - ACCV 2014, 2014, to appear.

    Open/Close abstract Abstract Info Info pdf PDF
  • A. Colomé, G. Neumann, J. Peters and C. Torras. Dimensionality reduction for probabilistic movement primitives, 2014 IEEE-RAS International Conference on Humanoid Robots, 2014, Madrid, Spain, pp. 794-800, IEEE Press.

    Open/Close abstract Abstract Info Info pdf PDF
  • A. Penate-Sanchez, F. Moreno-Noguer, J. Andrade-Cetto and F. Fleuret. LETHA: Learning from high quality inputs for 3D pose estimation in low quality images, 2014 International Conference on 3D Vision, 2014, Tokyo, to appear.

    Open/Close abstract Abstract Info Info pdf PDF
  • D. Martínez, G. Alenyà and C. Torras. Finding safe policies in model-based active learning, 2014 IROS Machine Learning in Planning and Control of Robot Motion Workshop, 2014, Chicago, pp. 1-6.

    Open/Close abstract Abstract Info Info pdf PDF
  • T.R. Savarimuthu, A.G. Buch, Y. Yang, W. Mustafar, S. Haller, J. Papon, D. Martínez and E.E. Aksoy. Manipulation monitoring and robot intervention in complex manipulation sequences, 2014 RSS Workshop on Robotic Monitoring, 2014, Berkeley, CA, USA.

    Open/Close abstract Abstract Info Info pdf PDF
  • F. Husain, B. Dellen and C. Torras. Recognizing point clouds using conditional random fields, 22nd International Conference on Pattern Recognition, 2014, Stockhom, Sweden, pp. 4257-4262, IEEE Computer Society.

    Open/Close abstract Abstract Info Info pdf PDF
  • F. Martí and G. Alenyà. Rigid and deformable pick and place algorithms. Technical Report IRI-TR-14-04, Institut de Robòtica i Informàtica Industrial, CSIC-UPC, 2014.

    Open/Close abstract Abstract Info Info pdf PDF
  • E. Simo-Serra, C. Torras and F. Moreno-Noguer. Geodesic finite mixture models, 2014 British Machine Vision Conference, 2014, .

    Open/Close abstract Abstract Info Info pdf PDF
  • F. Husain, B. Dellen and C. Torras. Robust surface tracking in range image sequences. Digital Signal Processing, 35: 37-44, 2014.

    Open/Close abstract Abstract Info Info pdf PDF
  • X. Solé, A. Ramisa and C. Torras. Evaluation of random forests on large-scale classification problems using a bag-of-visual-words representation, 17th Catalan Conference on Artificial Intelligence, 2014, Barcelona, in Artificial Intelligence Research and Development, Vol 269 of Frontiers in Artificial Intelligence and Applications, pp. 273-276, 2014, IOS Press.

    Open/Close abstract Abstract Info Info pdf PDF
  • W. Kazmi, S. Foix, G. Alenyà and H.J. Andersen. Indoor and outdoor depth imaging of leaves with time-of-flight and stereo vision sensors: Analysis and comparison. ISPRS Journal of Photogrammetry and Remote Sensing, 88: 128-146, 2014.

    Open/Close abstract Abstract Info Info pdf PDF
  • A. Agostini, C. Torras and F. Wörgötter. Learning weakly correlated cause-effects for gardening with a cognitive system. Engineering Applications of Artificial Intelligence, 36: 178–194, 2014.

    Open/Close abstract Abstract Info Info pdf PDF
  • E. Simo-Serra and A. Perez-Gracia. Kinematic synthesis using tree topologies. Mechanism and Machine Theory, 72: 94-113, 2014.

    Open/Close abstract Abstract Info Info pdf PDF
  • J. Borràs Sol, F. Thomas and C. Torras. New geometric approaches to the analysis and design of Stewart-Gough platforms. IEEE/ASME Transactions on Mechatronics, 19(2): 445-455, 2014.

    Open/Close abstract Abstract Info Info pdf PDF
  • F. Husain, A. Colomé, B. Dellen, G. Alenyà and C. Torras. Realtime tracking and grasping of a moving object from range video, 2014 IEEE International Conference on Robotics and Automation, 2014, Hong Kong, pp. 2617-2622.

    Open/Close abstract Abstract Info Info pdf PDF
  • C. Torras. Social robots: A meeting point between science and fiction. Mètode Science Studies Journal, 82, 2014.

    Open/Close abstract Abstract Info Info pdf PDF
  • D. Martínez, G. Alenyà, P. Jiménez, C. Torras, J. Rossmann, N. Wantia, A. Eren Erdal, S. Haller and J. Piater. Active learning of manipulation sequences, 2014 IEEE International Conference on Robotics and Automation, 2014, Hong Kong, pp. 5671-5678.

    Open/Close abstract Abstract Info Info pdf PDF
  • C. Torras. Learning algorithms for robot manipulation of clothing and plant leaves, 2014 ICRA Workshop on Advances in Robot Manipulation of Clothes and Flexible Objects, 2014, Hong-Kong.

    Open/Close abstract Abstract Info Info pdf PDF
  • G. Alenyà, J.L. Rivero, A. Rull, P. Grosch and S. Hernández. The HumanoidLab: Involving students in a research centre through an educational initiative, 6th International Conference on Computer Supported Education, 2014, Barcelona, pp. 213-220, Scitepress.

    Open/Close abstract Abstract Info Info pdf PDF
  • C. Torras. Robot manipulation in human environments: Challenges for learning algorithms, 14081 Dagstuhl Seminar, 2014, Dagstuhl, Germany, in Robots Learning from Experiences, Vol 4-2 of Dagstuhl Reports, pp. 99-101, Schloss Dagstuhl.

    Open/Close abstract Abstract Info Info pdf PDF
  • Y. Isleyici and G. Alenyà. Teaching grasping points using natural movements. Technical Report IRI-TR-14-01, Institut de Robòtica i Informàtica Industrial, CSIC-UPC, 2014.

    Open/Close abstract Abstract Info Info pdf PDF
  • L. Ferraz, X. Binefa and F. Moreno-Noguer. Very fast solution to the PnP problem with algebraic outlier rejection, 2014 IEEE Computer Society Conference on Computer Vision and Pattern Recognition, 2014, Columbus, Ohio (USA), pp. 501-508.

    Open/Close abstract Abstract Info Info pdf PDF
  • E. Trulls Fortuny, S. Tsogkas, I. Kokkinos, A. Sanfeliu and F. Moreno-Noguer. Segmentation-aware deformable part models, 2014 IEEE Computer Society Conference on Computer Vision and Pattern Recognition, 2014, Columbus, Ohio (USA), pp. 168-175.

    Open/Close abstract Abstract Info Info pdf PDF
  • A. Amor, A. Ruiz, F. Moreno-Noguer and A. Sanfeliu. On-board real-time pose estimation for UAVs using deformable visual contour registration, 2014 IEEE International Conference on Robotics and Automation, 2014, Hong Kong, pp. 2595-2601.

    Open/Close abstract Abstract Info Info pdf PDF
  • G. Alenyà, S. Foix and C. Torras. ToF cameras for active vision in robotics. Sensors and Actuators A: Physical, 218: 10-22, 2014.

    Open/Close abstract Abstract Info Info pdf PDF
  • M. Villamizar, A. Sanfeliu and F. Moreno-Noguer. Fast online learning and detection of natural landmarks for autonomous aerial robots, 2014 IEEE International Conference on Robotics and Automation, 2014, Hong Kong, pp. 4996-5003.

    Open/Close abstract Abstract Info Info pdf PDF
  • A. Ramisa, G. Alenyà, F. Moreno-Noguer and C. Torras. Learning RGB-D descriptors of garment parts for informed robot grasping. Engineering Applications of Artificial Intelligence, 35: 246-258, 2014.

    Open/Close abstract Abstract Info Info pdf PDF
  • A. Rubio, M. Villamizar, L. Ferraz, A. Penate-Sanchez, A. Sanfeliu and F. Moreno-Noguer. Estimación monocular y eficiente de la pose usando modelos 3D complejos, XXXV Jornadas de Automática, 2014, Valencia .

    Open/Close abstract Abstract Info Info pdf PDF

Perception and Manipulation Laboratory

The Perception and Manipulation Laboratory is equipped with 2 workcells: one with 2 standard manipulators and an XY positioner, and the other with 2 WAM arms in a reconfigurable arrangement. Additionally, researchers can find a 3 fingered hand, a Delta haptic interface, force sensors, several conventional cameras, and high speed, high resolution, and 3D cameras. Laboratory service offers quick experimental setup, several standardized software tools, and expertise in robot control and perception algorithms. It also hosts the Humanoid Lab initiative, with 15 small humanoid robots.

Perception and Manipulation Laboratory
Group photo

Researchers

PhD Students

Master Students

PFC Students

Support Staff