Research line

Perception and Manipulation Image

The research of PERCEPTION AND MANIPULATION group focuses on enhancing the perception, learning, and planning capabilities of robots to achieve higher degrees of autonomy and user-friendliness during everyday manipulation tasks. Some topics addressed are the geometric interpretation of perceptual information, construction of 3D object models, action selection and planning, reinforcement learning, and teaching by demonstration.

Head of line: Carme Torras Genís

Head of line

Tech. transfer

Our activity finds applications in several fields through collaboration with our technological partners

Research projects

We carry out projects from national and international research programmes.
→ More about our research projects

<< Back to Perception and Manipulation main page

Learning by demonstration

We devise methods to learn object-action relations to accomplish tasks at different levels of abstraction, where object models are generated from visual and depth information, and actions, involving manipulation skills, are learned from demonstrations provided by a human using multimodal algorithms that combine vision and haptics.

→ more info

Research area 1 of Perception and Manipulation

Planning for perception and manipulation

We are interested in view planning for object modeling, as well as manipulation planning, with special interest in deformable objects. High-level task formulations are integrated with low-level geometry-based methods and simplified physical models, together with an on-line sensory-based treatment of uncertainty, so as to come up with specific sequences of motion commands.

→ more info

Research area 2 of Perception and Manipulation

Perception of rigid and non-rigid objects

Our objective is to investigate computer vision algorithms for interpreting and understanding scenes from images, with applications in robotics and medical imaging. In particular, our activities are concentrated on retrieving rigid and non-rigid shape, motion and camera pose from single images and video sequences.

→ more info

Research area 3 of Perception and Manipulation

These are the latest research projects of the Perception and Manipulation research line:

These are the most recent publications (2017 - 2016) of the Perception and Manipulation

  • M. Villamizar, A. Garrell Zulueta, A. Sanfeliu and F. Moreno-Noguer. Random clustering ferns for multimodal object recognition. Neural Computing and Applications, 2017, to appear.

    Open/Close abstract Abstract Info Info pdf PDF
  • A. Agudo and F. Moreno-Noguer. Combining local-physical and global-statistical models for sequential deformable shape from motion. International Journal of Computer Vision, 122(2): 371-387, 2017.

    Open/Close abstract Abstract Info Info pdf PDF
  • A. Agudo and F. Moreno-Noguer. Global model with local interpretation for dynamic shape reconstruction, 2017 IEEE Winter Conference on Applications of Computer Vision, 2017, Santa Rosa (California), USA, IEEE, to appear.

    Open/Close abstract Abstract Info Info pdf PDF
  • S. Bensch, A. Jevtić and T. Hellström. On interaction quality in human-robot interaction, 2017 International Conference on Agents and Artificial Intelligence, 2017, Porto, Portugal, pp. 182-189, to appear.

    Open/Close abstract Abstract Info Info pdf PDF
  • E. Simo-Serra, C. Torras and F. Moreno-Noguer. 3D human pose tracking priors using geodesic mixture models. International Journal of Computer Vision, 2017, to appear.

    Open/Close abstract Abstract Info Info pdf PDF
  • A. Garrell Zulueta, M. Villamizar, F. Moreno-Noguer and A. Sanfeliu. Teaching robot’s proactive behavior using human assistance. International Journal of Social Robotics, 2017, to appear.

    Open/Close abstract Abstract Info Info pdf PDF
  • A. Pumarola, A. Vakhitov, A. Agudo, A. Sanfeliu and F. Moreno-Noguer. PL-SLAM: Real-time monocular visual SLAM with points and lines, 2017 IEEE International Conference on Robotics and Automation, 2017, Singapore, to appear.

    Open/Close abstract Abstract Info Info pdf PDF
  • M. Villamizar, J. Andrade-Cetto, A. Sanfeliu and F. Moreno-Noguer. Boosted random ferns for object detection. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2017, to appear.

    Open/Close abstract Abstract Info Info pdf PDF
  • A. Colomé and C. Torras. Dual REPS: A generalization of relative entropy policy search exploiting bad experiences. IEEE Transactions on Robotics, 2017, to appear.

    Open/Close abstract Abstract Info Info pdf PDF
  • M. Alberich-Carramiñana, B. Elizalde and F. Thomas. New algebraic conditions for the identification of the relative position of two coplanar ellipses. Computer Aided Geometric Design, 2017, to appear.

    Open/Close abstract Abstract Info Info pdf PDF
  • D. Martínez, G. Alenyà and C. Torras. Relational reinforcement learning with guided demonstrations. Artificial Intelligence, 2017, to appear.

    Open/Close abstract Abstract Info Info pdf PDF
  • A. Agostini, C. Torras and F. Wörgötter. Efficient interactive decision-making framework for robotic applications. Artificial Intelligence, 2017, to appear.

    Open/Close abstract Abstract Info Info pdf PDF
  • A. Agudo, J.M. Martínez, L. Agapito and B. Calvo. Modal space: A physics-based model for sequential estimation of time-varying shape from monocular video. Journal of Mathematical Imaging and Vision, 57(1): 75–98, 2017.

    Open/Close abstract Abstract Info Info pdf PDF
  • E. Trulls Fortuny, I. Kokkinos, A. Sanfeliu and F. Moreno-Noguer. Dense segmentation-aware descriptors. In Dense Image Correspondences for Computer Vision, 83-107. Springer, 2016.

    Open/Close abstract Abstract Info Info pdf PDF
  • A. Garcia, S. Foix and G. Alenyà. Construcció i programació d'un cap robòtic. Technical Report IRI-TR-16-03, Institut de Robòtica i Informàtica Industrial, CSIC-UPC, 2016.

    Open/Close abstract Abstract Info Info pdf PDF
  • G. Sanromà, A. Penate-Sanchez, R. Alquézar Mancho, F. Serratosa, F. Moreno-Noguer, J. Andrade-Cetto and M.A. González. MSClique: Multiple structure discovery through the maximum weighted clique problem. PLOS One, 11(1): e0145846, 2016.

    Open/Close abstract Abstract Info Info pdf PDF
  • M. Villamizar, A. Garrell Zulueta, A. Sanfeliu and F. Moreno-Noguer. Interactive multiple object learning with scanty human supervision. Computer Vision and Image Understanding, 149: 51-64, 2016.

    Open/Close abstract Abstract Info Info pdf PDF
  • M. Alberich-Carramiñana and V. González-Alonso. Determining plane curve singularities from its polars. Advances in Mathematics , 287(1): 788-822, 2016.

    Open/Close abstract Abstract Info Info pdf PDF
  • M. Arduengo, G. Alenyà and F. Moreno-Noguer. Database for 3D human pose estimation from single depth images. Technical Report IRI-TR-16-05, Institut de Robòtica i Informàtica Industrial, CSIC-UPC, 2016.

    Open/Close abstract Abstract Info Info pdf PDF
  • C. Torras. Service robots for citizens of the future. European Review, 24(1): 17-30, 2016.

    Open/Close abstract Abstract Info Info pdf PDF
  • F. Moreno-Noguer and J.M. Porta. A Bayesian approach to simultaneously recover camera pose and non-rigid shape from monocular images. Image and Vision Computing, 52: 141-153, 2016.

    Open/Close abstract Abstract Info Info pdf PDF
  • A. Agudo, J.M. Martínez, B. Calvo and F. Moreno-Noguer. Mode-shape interpretation: Re-thinking modal space for recovering deformable shapes, 2016 IEEE Winter Conference on Applications of Computer Vision, 2016, Lake Placid, USA, pp. 1-8, IEEE.

    Open/Close abstract Abstract Info Info pdf PDF
  • G. Canal, G. Alenyà and C. Torras. Personalization framework for adaptive robotic feeding assistance, 8th International Conference on Social Robotics, 2016, Kansas City, USA, in Social Robotics, Vol 9979 of Lecture Notes in Artificial Intelligence, pp. 22-31, 2016, Springer.

    Open/Close abstract Abstract Info Info pdf PDF
  • D. Martínez, G. Alenyà, C. Torras, T. Ribeiro and K. Inoue. Learning relational dynamics of stochastic domains for planning, 26th International Conference on Automated Planning and Scheduling, 2016, London, pp. 235-243.

    Open/Close abstract Abstract Info Info pdf PDF
  • A. Jevtić, A. Colomé, G. Alenyà and C. Torras. User evaluation of an interactive learning framework for single-arm and dual-arm robots, 8th International Conference on Social Robotics, 2016, Kansas City, USA, in Social Robotics, Vol 9979 of Lecture Notes in Artificial Intelligence, pp. 52-61, 2016, Springer.

    Open/Close abstract Abstract Info Info pdf PDF
  • G. Canal, S. Escalera and C. Angulo. A real-time human-robot interaction system based on gestures for assistive scenarios. Computer Vision and Image Understanding, 149: 65-77, 2016.

    Open/Close abstract Abstract Info Info pdf PDF
  • M. Alberich-Carramiñana, F. Dachs and J. Àlvarez. Multiplier ideals in two-dimensional local rings with rational singularities. Michigan Mathematical Journal, 65(2): 287-320, 2016.

    Open/Close abstract Abstract Info Info pdf PDF
  • J.G. Hoyos, F. Prieto, G. Alenyà and C. Torras. Execution Fault Recovery in Robot Programming by Demonstration Using Multiple Models. IEEE Latin America Transactions , 14(2): 517-523, 2016.

    Open/Close abstract Abstract Info Info pdf PDF
  • G. Martín, F. Husain, H. Schulz, S. Frintrop, C. Torras and S. Behnke. Semantic segmentation priors for object discovery, 23rd International Conference on Pattern Recognition, 2016, Cancún, Mexico, to appear.

    Open/Close abstract Abstract Info Info pdf PDF
  • A. Agudo, F. Moreno-Noguer, B. Calvo and J.M. Martínez. Real-time 3D reconstruction of non-rigid shapes with a single moving camera. Computer Vision and Image Understanding, 153(12): 37–54, 2016.

    Open/Close abstract Abstract Info Info pdf PDF
  • A. Rubio, L. Yu, E. Simo-Serra and F. Moreno-Noguer. BASS: Boundary-aware superpixel segmentation, 23rd International Conference on Pattern Recognition, 2016, Cancún, Mexico, to appear.

    Open/Close abstract Abstract Info Info pdf PDF
  • F. Husain, H. Schulz, B. Dellen, C. Torras and S. Behnke. Combining semantic and geometric features for object class segmentation of indoor scenes. IEEE Robotics and Automation Letters, 2(1): 49-55, 2016.

    Open/Close abstract Abstract Info Info pdf PDF
  • A. Vakhitov, J. Funke and F. Moreno-Noguer. Accurate and linear time pose estimation from points and lines, 14th European Conference on Computer Vision, 2016, Amsterdam, in Computer Vision - ECCV 2016, Vol 9911 of Lecture Notes in Computer Science, pp. 583-599, 2016.

    Open/Close abstract Abstract Info Info pdf PDF
  • C. Torras. Robot pain: A speculative review of its functions. In Pain and the Conscious Brain, 235-246. Wolters Kluwer, 2016.

    Open/Close abstract Abstract Info Info pdf PDF
  • J. Funke, J. Klein, F. Moreno-Noguer, A. Cardona and M. Cook. Structured learning of assignment models for neuron reconstruction to minimize topological errors, 13th IEEE International Symposium on Biomedical Imaging, 2016, Prague, pp. 607-611.

    Open/Close abstract Abstract Info Info pdf PDF
  • F. Husain, B. Dellen and C. Torras. Action recognition based on efficient deep feature learning in the spatio-temporal domain. IEEE Robotics and Automation Letters, 1(2): 984-991, 2016.

    Open/Close abstract Abstract Info Info pdf PDF
  • J. Buhmann, S. Gerhard, M. Cook and J. Funke. Tracking of microtubules in anisotropic volumes of neural tissue, 13th IEEE International Symposium on Biomedical Imaging, 2016, Prague, pp. 326-329.

    Open/Close abstract Abstract Info Info pdf PDF
  • L. Rozo, S. Calinon, D. Caldwell, P. Jiménez and C. Torras. Learning physical collaborative robot behaviors from human demonstrations. IEEE Transactions on Robotics, 32(3): 513-527, 2016.

    Open/Close abstract Abstract Info Info pdf PDF
  • A. Quattoni, A. Ramisa, P. Swaroop, E. Simo-Serra and F. Moreno-Noguer. Structured prediction with output embeddings for semantic image annotation, 2016 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies , 2016, San Diego, pp. 552-557.

    Open/Close abstract Abstract Info Info pdf PDF
  • N. Covallero and G. Alenyà. Grasping novel objects. Technical Report IRI-TR-16-01, Institut de Robòtica i Informàtica Industrial, CSIC-UPC, 2016.

    Open/Close abstract Abstract Info Info pdf PDF
  • A. Agudo and F. Moreno-Noguer. Recovering pose and 3D deformable shape from multi-instance image ensembles, 13th Asian Conference on Computer Vision, 2016, Taipei, Taiwan, in Computer Vision – ACCV 2016, Vol 10114 of Lecture Notes in Computer Science, pp. 291-307, 2017, Springer.

    Open/Close abstract Abstract Info Info pdf PDF
  • J.G. Hoyos, F. Prieto, G. Alenyà and C. Torras. Incremental learning of skills in a task-parameterized Gaussian mixture model. Journal of Intelligent and Robotic Systems, 82(1): 81-99, 2016.

    Open/Close abstract Abstract Info Info pdf PDF
  • A. Ramisa, G. Alenyà, F. Moreno-Noguer and C. Torras. A 3D descriptor to detect task-oriented grasping points in clothing. Pattern Recognition, 60: 936-948, 2016.

    Open/Close abstract Abstract Info Info pdf PDF
  • A. Agudo, F. Moreno-Noguer, B. Calvo and J.M. Martínez. Sequential non-rigid structure from motion using physical priors. IEEE Transactions on Pattern Analysis and Machine Intelligence, 38(5): 979-994, 2016.

    Open/Close abstract Abstract Info Info pdf PDF
  • A. Gabás, E. Corona, G. Alenyà and C. Torras. Robot-aided cloth classification using depth information and CNNs, 9th Conference on Articulated Motion and Deformable Objects, 2016, Palma de Mallorca, Spain, in Articulated Motion of Deformable Objects, Vol 9756 of Lecture Notes in Computer Science, pp. 16-23, 2016, Springer.

    Open/Close abstract Abstract Info Info pdf PDF

Perception and Manipulation Laboratory

The Perception and Manipulation Laboratory is equipped with 2 workcells: one with 2 standard manipulators and an XY positioner, and the other with 2 WAM arms in a reconfigurable arrangement. Additionally, researchers can find a 3 fingered hand, a Delta haptic interface, force sensors, several conventional cameras, and high speed, high resolution, and 3D cameras. Laboratory service offers quick experimental setup, several standardized software tools, and expertise in robot control and perception algorithms. It also hosts the Humanoid Lab initiative, with 15 small humanoid robots.

Perception and Manipulation Laboratory
Group photo

Researchers

PhD Students

Master Students

PFC Students