Research Project

IntellAct: Intelligent observation and execution of Actions and manipulations

Type

European Project

Start Date

01/03/2011

End Date

28/02/2014

Project Code

FP7-ICT2009-6-269959

Project illustration

Staff

Project Description

IntellAct addresses the problem of understanding and exploiting the meaning (semantics) of manipulations in terms of objects, actions and their consequences for reproducing human actions with machines. It comprises three building blocks: (1) Learning: Abstract, semantic descriptions of manipulations are extracted from video sequences showing a human demonstrating the manipulations; (2) Monitoring: In the second step, observed manipulations are evaluated against the learned, semantic models; (3) Execution: Based on learned, semantic models, equivalent manipulations are executed by a robot. The analysis of low-level observation data for semantic content (Learning) and the synthesis of concrete behavior (Execution) constitute the major scientific challenge of IntellAct. Based on the semantic interpretation and description and enhanced with low-level trajectory data for grounding, two major application areas are addressed by IntellAct: First, the monitoring of human manipulations for correctness (e.g., for training or in high-risk scenarios) and second, the efficient teaching of cognitive robots for manipulation (e.g. in the context of assembly processes in industrial settings and on a space station). To achieve these goals, IntellAct brings together recent methods for (1) parsing scenes into spatio-temporal graphs and so called ‘semantic event chains’, (2) probabilistic models of objects and their manipulation, (3) probabilistic rule learning, and (4) dynamic motion primitives for trainable and flexible descriptions of robotic motor behavior. Its implementation employs a concurrent-engineering approach that includes virtual-reality-enhanced simulation as well as physical robots. Its goal culminates in the demonstration of a robot understanding, monitoring and reproducing human action.

Project Publications

Journal Publications

  • L. Rozo, P. Jiménez and C. Torras. A robot learning from demonstration framework to perform force-based manipulation tasks. Intelligent Service Robotics, 6(1): 33-51, 2013.

    Open/Close abstract Abstract Info Info pdf PDF

  • A. Penate-Sanchez, J. Andrade-Cetto and F. Moreno-Noguer. Exhaustive linearization for robust camera pose and focal length estimation. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2013, to appear.

    Open/Close abstract Abstract Info Info pdf PDF

  • P. Jiménez. Survey on assembly sequencing: A combinatorial and geometrical perspective. Journal of Intelligent Manufacturing, 24(2): 235-250, 2013.

    Open/Close abstract Abstract Info Info pdf PDF

  • A. Abramov, K. Pauwels, J. Papon, F. Wörgötter and B. Dellen. Real-time segmentation of stereo videos on a portable system with a mobile GPU. IEEE Transactions on Circuits and Systems for Video Technology, 22(9): 1292-1305, 2012.

    Open/Close abstract Abstract Info Info pdf PDF

  • S. Ulbrich, V. Ruiz de Angulo, C. Torras, T. Asfour and R. Dillman. Kinematic Bézier maps. IEEE Transactions on Systems, Man and Cybernetics: Part B, 42(4): 1215-1230, 2012.

    Open/Close abstract Abstract Info Info pdf PDF

Conference Publications

  • B. Dellen, S.F. Husain and C. Torras. Joint segmentation and tracking of object surfaces in depth movies along human/robot manipulations, 8th International Conference on Computer Vision Theory and Applications, 2013, Barcelona, pp. 244-251, to appear.

    Open/Close abstract Abstract Info Info pdf PDF

  • A. Colomé, D. Pardo, G. Alenyà and C. Torras. External force estimation during compliant robot manipulation , 2013 IEEE International Conference on Robotics and Automation, 2013, , to appear.

    Open/Close abstract Abstract Info Info pdf PDF

  • G. Alenyà, A. Ramisa, F. Moreno-Noguer and C. Torras. Characterization of textile grasping experiments, 2012 ICRA Workshop on Conditions for Replicable Experiments and Performance Comparison in Robotics Research, 2012, St Paul, Minnesota, USA, pp. 1-6.

    Open/Close abstract Abstract Info Info pdf PDF

  • A. Colomé, D. Pardo, G. Alenyà and C. Torras. External force estimation for textile grasp detection, 2012 IROS Workshop Beyond Robot Grasping: Modern Approaches for Learning Dynamic Manipulation, 2012, Vilamoura, Portugal, pp. 1-1.

    Open/Close abstract Abstract Info Info pdf PDF

  • D. Pardo, L. Rozo, G. Alenyà and C. Torras. Dynamically consistent probabilistic model for robot motion learning, 2012 IROS Workshop on Learning and Interaction in Haptic Robots, 2012, Algarve, Portugal, pp. 1-2.

    Open/Close abstract Abstract Info Info pdf PDF

  • L. Rozo, P. Jiménez and C. Torras. Robot learning from demonstration in the force domain, 2011 IJCAI Workshop on Agents Learning Interactively from Human Teachers, 2011, Barcelona, Spain, pp. 1-6.

    Open/Close abstract Abstract Info Info pdf PDF

  • S. Reich, A. Abramov, J. Papon, F. Wörgötter and B. Dellen. A novel real-time edge-preserving smoothing filter, 8th International Conference on Computer Vision Theory and Applications, 2013, Barcelona, to appear.

    Open/Close abstract Abstract Info Info pdf PDF

  • L. Rozo, P. Jiménez and C. Torras. Robot learning from demonstration of force-based tasks with multiple solution trajectories, 15th International Conference on Advanced Robotics, 2011, Tallin, Estonia, pp. 124-129, IEEE.

    Open/Close abstract Abstract Info Info pdf PDF

  • E.E. Aksoy, B. Dellen, M. Tamosiunaite and F. Wörgötter. Execution of a dual-object (pushing) action with semantic event chains, 11th IEEE-RAS International Conference on Humanoid Robots, 2011, Bled, Slovenia, pp. 576-583, IEEE Press.

    Open/Close abstract Abstract Info Info pdf PDF

  • A. Colomé, G. Alenyà and C. Torras. Handling high parameter dimensionality in reinforcement learning with dynamic motor primitives, 2013 ICRA Workshop on Novel Methods for Learning and Optimization of Control Policies and Trajectories for Robotics, 2013, Karlsruhe (Germany), to appear.

    Open/Close abstract Abstract Info Info

  • A.A. Ortega and J. Andrade-Cetto. Segmentation of dynamic objects from laser data, 5th European Conference on Mobile Robots, 2011, Örebro, Sweden, pp. 115-121, Örebro Univ.

    Open/Close abstract Abstract Info Info pdf PDF

  • A. Colomé and C. Torras. Positioning two redundant arms for cooperative manipulation of objects, 2013 International Workshop on Computational Kinematics, 2013, Barcelona, to appear.

    Open/Close abstract Abstract Info Info pdf PDF

  • E.H. Teniente and J. Andrade-Cetto. FaMSA: Fast multi-scan alignment with partially known correspondences, 5th European Conference on Mobile Robots, 2011, Örebro, Sweden, pp. 139-144, Örebro Univ.

    Open/Close abstract Abstract Info Info pdf PDF

  • A. Alexey, K. Pauwels, J. Papon, F. Wörgötter and B. Dellen. Depth-supported real-time video segmentation with the Kinect, 2012 IEEE Workshop on Applications of Computer Vision, 2012, Breckenridge, CO, USA, pp. 457-464.

    Open/Close abstract Abstract Info Info pdf PDF

  • E.H. Teniente, R. Valencia and J. Andrade-Cetto. Dense outdoor 3D mapping and navigation with Pose SLAM, 2011 Workshop de Robótica Experimental, 2011, Seville, pp. 567-572.

    Open/Close abstract Abstract Info Info pdf PDF

  • A. Ramisa, G. Alenyà, F. Moreno-Noguer and C. Torras. Using depth and appearance features for informed robot grasping of highly wrinkled clothes, 2012 IEEE International Conference on Robotics and Automation, 2012, St. Paul, Minnesota, USA, pp. 1703-1708, IEEE.

    Open/Close abstract Abstract Info Info pdf PDF

  • S. Foix, S. Kriegel, S. Fuchs, G. Alenyà and C. Torras. Information-gain view planning for free-form object reconstruction with a 3D ToF camera, 14th International Conference on Advanced Concepts for Intelligent Vision Systems, 2012, Brno, Czech Republic, in Advanced Concepts for Intelligent Vision Systems, Vol 7517 of Lecture Notes in Computer Science, pp. 36-47, 2012, Springer.

    Open/Close abstract Abstract Info Info pdf PDF

  • F. Rigual, A. Ramisa, G. Alenyà and C. Torras. Object detection methods for robot grasping: Experimental assessment and tuning, 15th International Conference of the Catalan Association for Artificial Intelligence, 2012, Alacant, in Artificial Intelligence Research and Development, Vol 248 of Frontiers in Artificial Intelligence and Applications, pp. 123-132, 2012, IOS Press.

    Open/Close abstract Abstract Info Info pdf PDF

  • E. Simo-Serra, A. Ramisa, G. Alenyà, C. Torras and F. Moreno-Noguer. Single image 3D human pose estimation from noisy observations, 2012 IEEE Computer Society Conference on Computer Vision and Pattern Recognition, 2012, Providence, Rhode Island, pp. 2673-2680, IEEE Press.

    Open/Close abstract Abstract Info Info pdf PDF

  • A. Colomé and C. Torras. Redundant inverse kinematics: Experimental comparative review and two enhancements, 2012 IEEE/RSJ International Conference on Intelligent Robots and Systems, 2012, Vilamoura, Portugal, pp. 5333-5340, IEEE.

    Open/Close abstract Abstract Info Info pdf PDF

  • P. Monsó, G. Alenyà and C. Torras. POMDP approach to robotized clothes separation, 2012 IEEE/RSJ International Conference on Intelligent Robots and Systems, 2012, Vilamoura, Portugal, pp. 1324-1329, IEEE.

    Open/Close abstract Abstract Info Info pdf PDF