IRI - GARNICS: Gardening with a cognitive system

The GARNICS project aims at 3D sensing of plant growth and building perceptual representations for learning the links to actions of a robot gardener. Plants are complex, self-changing systems with increasing complexity over time. Actions performed at plants (like watering), will have strongly delayed effects. Thus, monitoring and controlling plants is a difficult perception-action problem requiring advanced predictive cognitive properties, which so far can only be provided by experienced human gardeners. Sensing and control of a plants actual properties, i.e. its phenotype, is relevant to e.g. seed production and plant breeders.

We address plant sensing and control by combining active vision with appropriate perceptual representations, which are essential for cognitive interactions. Core ingredients for these representations are channel representations, dynamic graphs and cause-effect couples (CECs). Channel representations are a wavelet-like, biologically motivated information representation, which can be generalized coherently using group theory. Using these representations, plant models (represented by dynamic graphs) will be acquired, and, by interacting with a human gardener, the system will be taught the different cause-effect relations resulting from possible treatments. Employing decision making and planning processes via CECs, our robot gardener will then be able to choose from its learned repertoire the appropriate actions for optimal plant growth. This way we will arrive at an adaptive, interactive cognitive system, which will be implemented and tested in an industrially-relevant plant-phenotyping application.

http://www.garnics.eu/

Project Publications

Journal Publications

S. Foix, G. Alenyà and C. Torras. Task-driven active sensing framework applied to leaf probing. Computers and Electronics in Agriculture, 147: 166-175, 2018.

Abstract Info PDF
A. Agostini, G. Alenyà, A. Fischbach, H. Scharr, F. Wörgötter and C. Torras. A cognitive architecture for automatic gardening. Computers and Electronics in Agriculture, 138: 69-79, 2017.

Abstract Info PDF
C. Torras. Service robots for citizens of the future. European Review, 24(1): 17-30, 2016.

Abstract Info PDF
B. Dellen, H. Scharr and C. Torras. Growth signature of rosette plants from time-lapse video. IEEE/ACM Transactions on Computational Biology and Bioinformatics, 12(6): 1470-1478, 2015.

Abstract Info PDF
E.E. Aksoy, A. Abramov, F. Wörgötter, H. Scharr, A. Fischbach and B. Dellen. Modeling leaf growth of rosette plants using infrared stereo image sequences. Computers and Electronics in Agriculture, 110: 78-90, 2015.

Abstract Info PDF
W. Kazmi, S. Foix, G. Alenyà and H.J. Andersen. Indoor and outdoor depth imaging of leaves with time-of-flight and stereo vision sensors: Analysis and comparison. ISPRS Journal of Photogrammetry and Remote Sensing, 88: 128-146, 2014.

Abstract Info PDF
G. Alenyà, S. Foix and C. Torras. ToF cameras for active vision in robotics. Sensors and Actuators A: Physical, 218: 10-22, 2014.

Abstract Info PDF
A. Agostini, C. Torras and F. Wörgötter. Learning weakly correlated cause-effects for gardening with a cognitive system. Engineering Applications of Artificial Intelligence, 36: 178–194, 2014.

Abstract Info PDF
F. Moreno-Noguer and P. Fua. Stochastic exploration of ambiguities for nonrigid shape recovery. IEEE Transactions on Pattern Analysis and Machine Intelligence, 35(2): 463-475, 2013.

Abstract Info PDF
G. Alenyà, B. Dellen, S. Foix and C. Torras. Robotized plant probing: Leaf segmentation utilizing time-of-flight data. IEEE Robotics and Automation Magazine, 20(3): 50-59, 2013.

Abstract Info PDF
E. Todt and C. Torras. Outdoor view recognition based on landmark grouping and logistic regression. International Journal of Pattern Recognition and Artificial Intelligence, 27(3): 1-21, 2013.

Abstract Info PDF
S. Ulbrich, V. Ruiz de Angulo, T. Asfour, C. Torras and R. Dillman. General robot kinematics decomposition without intermediate markers. IEEE Transactions on Neural Networks, 23(4): 620-630, 2012.

Abstract Info PDF
E.E. Aksoy, A. Abramov, J. Dörr, K. Ning, B. Dellen and F. Wörgötter. Learning the semantics of object-action relations by observation. The International Journal of Robotics Research, 30(10): 1229-1249, 2011.

Abstract Info PDF

Conference Publications

C. Torras. Learning algorithms for robot manipulation of clothing and plant leaves, 2014 ICRA Workshop on Advances in Robot Manipulation of Clothes and Flexible Objects, 2014, Hong-Kong.

Abstract Info PDF
C. Torras. From the Turing test to science fiction: The challenges of social robotics, 16th Catalan Conference on Artificial Intelligence, 2013, Vic, Spain, in Artificial Intelligence Research and Development, Vol 256 of Frontiers in Artificial Intelligence and Applications, pp. 5-7, 2013, IOS Press.

Abstract Info PDF
B. Dellen, F. Husain and C. Torras. Joint segmentation and tracking of object surfaces in depth movies along human/robot manipulations, 8th International Conference on Computer Vision Theory and Applications, 2013, Barcelona, pp. 244-251.

Abstract Info PDF
E. Serradell, P. Glowacki, K. Jan, F. Moreno-Noguer and P. Fua. Robust non-rigid registration of 2D and 3D graphs, 2012 IEEE/CVF Conference on Computer Vision and Pattern Recognition, 2012, Providence, RI, USA, pp. 996-1003, IEEE Press.

Abstract Info PDF
E. Serradell, K. Jan, F. Moreno-Noguer and P. Fua. Robust elastic 2D/3D geometric graph matching, 2012 SPIE Medical Imaging: Image Processing, 2012, San Diego CA, USA, Vol 8314 of Proceedings of the SPIE, 2012, SPIE.

Abstract Info PDF
A. Alexey, K. Pauwels, J. Papon, F. Wörgötter and B. Dellen. Depth-supported real-time video segmentation with the Kinect, 2012 IEEE Workshop on Applications of Computer Vision, 2012, Breckenridge, CO, USA, pp. 457-464.

Abstract Info PDF
A. Colomé and C. Torras. Redundant inverse kinematics: Experimental comparative review and two enhancements, 2012 IEEE/RSJ International Conference on Intelligent Robots and Systems, 2012, Vilamoura, Portugal, pp. 5333-5340, IEEE.

Abstract Info PDF
W. Kazmi, S. Foix and G. Alenyà. Plant leaf imaging using time of flight camera under sunlight, shadow and room conditions, 2012 IEEE International Symposium on Robotic and Sensors Environments, 2012, Magdeburg, Germany, pp. 192-197, IEEE.

Abstract Info PDF
G. Alenyà, B. Dellen, S. Foix and C. Torras. Robotic leaf probing via segmentation of range data into surface patches, 2012 IROS Workshop on Agricultural Robotics: Enabling Safe, Efficient, Affordable Robots for Food Production, 2012, Vilamoura, Portugal, pp. 1-6.

Abstract Info PDF
L. Rozo, P. Jiménez and C. Torras. Robot learning from demonstration of force-based tasks with multiple solution trajectories, 15th International Conference on Advanced Robotics, 2011, Tallin, Estonia, pp. 124-129, IEEE.

Abstract Info PDF
M. Wallenberg, M. Felsberg, P. Forssen and B. Dellen. Channel coding for joint colour and depth segmentation, 33rd Annual Symposium of the German Association for Pattern Recognition, 2011, Frankfurt, Germany, in Pattern Recognition, Vol 6835 of Lecture Notes in Computer Science, pp. 306-315, 2011, Springer.

Abstract Info PDF
A. Agostini, C. Torras and F. Wörgötter. Integrating task planning and interactive learning for robots to work in human environments, 22nd International Joint Conference on Artificial Intelligence, 2011, Barcelona, pp. 2386-2391.

Abstract Info PDF
F. Moreno-Noguer and J.M. Porta. Probabilistic simultaneous pose and non-rigid shape recovery, 2011 IEEE/CVF Conference on Computer Vision and Pattern Recognition, 2011, Colorado Springs, CO, USA, pp. 1289-1296, IEEE Press.

Abstract Info PDF
E. Simo-Serra, F. Moreno-Noguer and A. Perez. Design of non-anthropomorphic robotic hands for anthropomorphic tasks, 2011 ASME International Design Engineering Technical Conference, 2011, Washington, DC, USA, pp. 377-386.

Abstract Info PDF
F. Moreno-Noguer. Deformation and illumination invariant feature point descriptor, 2011 IEEE/CVF Conference on Computer Vision and Pattern Recognition, 2011, Colorado Springs, CO, USA, pp. 1593-1600, IEEE Press.

Abstract Info PDF
S. Foix, G. Alenyà and C. Torras. Towards plant monitoring through next best view, 14th Catalan Conference on Artificial Intelligence, 2011, Lleida, in Artificial Intelligence Research and Development, Vol 232 of Frontiers in Artificial Intelligence and Applications, pp. 101-109, 2011, IOS Press.

Abstract Info PDF
G. Alenyà, F. Moreno-Noguer, A. Ramisa and C. Torras. Active perception of deformable objects using 3D cameras, 2011 Workshop de Robótica Experimental, 2011, Seville, pp. 434-440.

Abstract Info PDF
B. Dellen, G. Alenyà, S. Foix and C. Torras. Segmenting color images into surface patches by exploiting sparse depth data, 2011 IEEE Workshop on Applications of Computer Vision, 2011, Kona, Hawaii, pp. 591-598.

Abstract Info PDF
E. Serradell, A. Romero, R. Leta, C. Gatta and F. Moreno-Noguer. Simultaneous correspondence and non-rigid 3D reconstruction of the coronary tree from single X-ray images, 13th International Conference on Computer Vision, 2011, Barcelona, pp. 850-857, IEEE.

Abstract Info PDF
G. Alenyà, B. Dellen and C. Torras. 3D modelling of leaves from color and ToF data for robotized plant measuring, 2011 IEEE International Conference on Robotics and Automation, 2011, Shanghai, China, pp. 3408-3414, IEEE.

Abstract Info PDF
L. Rozo, P. Jiménez and C. Torras. Robot learning from demonstration in the force domain, 2011 IJCAI Workshop on Agents Learning Interactively from Human Teachers, 2011, Barcelona, Spain, pp. 1-6.

Abstract Info PDF
J. Sanchez, J. Ostlund, P. Fua and F. Moreno-Noguer. Simultaneous pose, correspondence and non-rigid shape, 2010 IEEE/CVF Conference on Computer Vision and Pattern Recognition, 2010, San Francisco, CA, USA, pp. 1189-1196.

Abstract Info PDF
M. Villamizar, F. Moreno-Noguer, J. Andrade-Cetto and A. Sanfeliu. Efficient rotation invariant object detection using boosted random Ferns, 2010 IEEE/CVF Conference on Computer Vision and Pattern Recognition, 2010, San Francisco, CA, USA, pp. 1038-1045.

Abstract Info PDF
M. Villamizar, F. Moreno-Noguer, J. Andrade-Cetto and A. Sanfeliu. Shared random Ferns for efficient detection of multiple categories, 20th International Conference on Pattern Recognition, 2010, Istanbul, Turkey, pp. 388-391.

Abstract Info PDF
L. Rozo, P. Jiménez and C. Torras. Learning force-based robot skills from haptic demonstration, 13th Catalan Conference on Artificial Intelligence, 2010, Espluga de Francolí, Spain, in Artificial Intelligence Research and Development, Vol 220 of Frontiers in Artificial Intelligence and Applications, pp. 331-340, 2010, IOS Press.

Abstract Info PDF
L. Rozo, P. Jiménez and C. Torras. Sharpening haptic inputs for teaching a manipulation skill to a robot, 1st IEEE International Conference on Applied Bionics and Biomechanics, 2010, Venice, pp. 331-340.

Abstract Info PDF
F. Moreno-Noguer, J.M. Porta and P. Fua. Exploring ambiguities for monocular non-rigid shape estimation, 11th European Conference on Computer Vision, 2010, Crete, Greece, in Computer Vision - ECCV 2010, Vol 6313 of Lecture Notes in Computer Science, pp. 370-383, 2010, Springer, Berlin.

Abstract Info PDF
E. Serradell, M. Özuysal, V. Lepetit, P. Fua and F. Moreno-Noguer. Combining geometric and appearance priors for robust homography estimation, 11th European Conference on Computer Vision, 2010, Crete, Greece, in Computer Vision - ECCV 2010, Vol 6313 of Lecture Notes in Computer Science, pp. 58-72, 2010, Springer, Berlin.

Abstract Info PDF

Other Publications

A. Agostini, C. Torras and F. Wörgötter. A general strategy for interactive decision-making in robotic platforms. Technical Report IRI-TR-11-01, Institut de Robòtica i Informàtica Industrial, CSIC-UPC, 2011.

Abstract Info PDF

Institut de Robòtica i Informàtica Industrial, CSIC-UPC
C/ Llorens i Artigas 4-6, 08028, Barcelona, Spain

Site map
Accessibility
About this web & cookies
Disclaimer

The activities of our institute are supported by:

Research Project

GARNICS: Gardening with a cognitive system

Type

Start Date

End Date

Project Code

Staff

Principal Investigator

Researcher

Researcher

Researcher

Researcher

Researcher

Researcher

Researcher

Researcher

Researcher

Researcher

Member

Project Description

Project Publications

Journal Publications

S. Foix, G. Alenyà and C. Torras. Task-driven active sensing framework applied to leaf probing. Computers and Electronics in Agriculture, 147: 166-175, 2018.

A. Agostini, G. Alenyà, A. Fischbach, H. Scharr, F. Wörgötter and C. Torras. A cognitive architecture for automatic gardening. Computers and Electronics in Agriculture, 138: 69-79, 2017.

C. Torras. Service robots for citizens of the future. European Review, 24(1): 17-30, 2016.

B. Dellen, H. Scharr and C. Torras. Growth signature of rosette plants from time-lapse video. IEEE/ACM Transactions on Computational Biology and Bioinformatics, 12(6): 1470-1478, 2015.

E.E. Aksoy, A. Abramov, F. Wörgötter, H. Scharr, A. Fischbach and B. Dellen. Modeling leaf growth of rosette plants using infrared stereo image sequences. Computers and Electronics in Agriculture, 110: 78-90, 2015.

W. Kazmi, S. Foix, G. Alenyà and H.J. Andersen. Indoor and outdoor depth imaging of leaves with time-of-flight and stereo vision sensors: Analysis and comparison. ISPRS Journal of Photogrammetry and Remote Sensing, 88: 128-146, 2014.

G. Alenyà, S. Foix and C. Torras. ToF cameras for active vision in robotics. Sensors and Actuators A: Physical, 218: 10-22, 2014.

A. Agostini, C. Torras and F. Wörgötter. Learning weakly correlated cause-effects for gardening with a cognitive system. Engineering Applications of Artificial Intelligence, 36: 178–194, 2014.

F. Moreno-Noguer and P. Fua. Stochastic exploration of ambiguities for nonrigid shape recovery. IEEE Transactions on Pattern Analysis and Machine Intelligence, 35(2): 463-475, 2013.

G. Alenyà, B. Dellen, S. Foix and C. Torras. Robotized plant probing: Leaf segmentation utilizing time-of-flight data. IEEE Robotics and Automation Magazine, 20(3): 50-59, 2013.

E. Todt and C. Torras. Outdoor view recognition based on landmark grouping and logistic regression. International Journal of Pattern Recognition and Artificial Intelligence, 27(3): 1-21, 2013.

S. Ulbrich, V. Ruiz de Angulo, T. Asfour, C. Torras and R. Dillman. General robot kinematics decomposition without intermediate markers. IEEE Transactions on Neural Networks, 23(4): 620-630, 2012.

E.E. Aksoy, A. Abramov, J. Dörr, K. Ning, B. Dellen and F. Wörgötter. Learning the semantics of object-action relations by observation. The International Journal of Robotics Research, 30(10): 1229-1249, 2011.

Conference Publications

C. Torras. Learning algorithms for robot manipulation of clothing and plant leaves, 2014 ICRA Workshop on Advances in Robot Manipulation of Clothes and Flexible Objects, 2014, Hong-Kong.

B. Dellen, F. Husain and C. Torras. Joint segmentation and tracking of object surfaces in depth movies along human/robot manipulations, 8th International Conference on Computer Vision Theory and Applications, 2013, Barcelona, pp. 244-251.

E. Serradell, P. Glowacki, K. Jan, F. Moreno-Noguer and P. Fua. Robust non-rigid registration of 2D and 3D graphs, 2012 IEEE/CVF Conference on Computer Vision and Pattern Recognition, 2012, Providence, RI, USA, pp. 996-1003, IEEE Press.

E. Serradell, K. Jan, F. Moreno-Noguer and P. Fua. Robust elastic 2D/3D geometric graph matching, 2012 SPIE Medical Imaging: Image Processing, 2012, San Diego CA, USA, Vol 8314 of Proceedings of the SPIE, 2012, SPIE.

A. Alexey, K. Pauwels, J. Papon, F. Wörgötter and B. Dellen. Depth-supported real-time video segmentation with the Kinect, 2012 IEEE Workshop on Applications of Computer Vision, 2012, Breckenridge, CO, USA, pp. 457-464.

A. Colomé and C. Torras. Redundant inverse kinematics: Experimental comparative review and two enhancements, 2012 IEEE/RSJ International Conference on Intelligent Robots and Systems, 2012, Vilamoura, Portugal, pp. 5333-5340, IEEE.

W. Kazmi, S. Foix and G. Alenyà. Plant leaf imaging using time of flight camera under sunlight, shadow and room conditions, 2012 IEEE International Symposium on Robotic and Sensors Environments, 2012, Magdeburg, Germany, pp. 192-197, IEEE.

G. Alenyà, B. Dellen, S. Foix and C. Torras. Robotic leaf probing via segmentation of range data into surface patches, 2012 IROS Workshop on Agricultural Robotics: Enabling Safe, Efficient, Affordable Robots for Food Production, 2012, Vilamoura, Portugal, pp. 1-6.

L. Rozo, P. Jiménez and C. Torras. Robot learning from demonstration of force-based tasks with multiple solution trajectories, 15th International Conference on Advanced Robotics, 2011, Tallin, Estonia, pp. 124-129, IEEE.

A. Agostini, C. Torras and F. Wörgötter. Integrating task planning and interactive learning for robots to work in human environments, 22nd International Joint Conference on Artificial Intelligence, 2011, Barcelona, pp. 2386-2391.

F. Moreno-Noguer and J.M. Porta. Probabilistic simultaneous pose and non-rigid shape recovery, 2011 IEEE/CVF Conference on Computer Vision and Pattern Recognition, 2011, Colorado Springs, CO, USA, pp. 1289-1296, IEEE Press.

E. Simo-Serra, F. Moreno-Noguer and A. Perez. Design of non-anthropomorphic robotic hands for anthropomorphic tasks, 2011 ASME International Design Engineering Technical Conference, 2011, Washington, DC, USA, pp. 377-386.

F. Moreno-Noguer. Deformation and illumination invariant feature point descriptor, 2011 IEEE/CVF Conference on Computer Vision and Pattern Recognition, 2011, Colorado Springs, CO, USA, pp. 1593-1600, IEEE Press.

S. Foix, G. Alenyà and C. Torras. Towards plant monitoring through next best view, 14th Catalan Conference on Artificial Intelligence, 2011, Lleida, in Artificial Intelligence Research and Development, Vol 232 of Frontiers in Artificial Intelligence and Applications, pp. 101-109, 2011, IOS Press.

G. Alenyà, F. Moreno-Noguer, A. Ramisa and C. Torras. Active perception of deformable objects using 3D cameras, 2011 Workshop de Robótica Experimental, 2011, Seville, pp. 434-440.

B. Dellen, G. Alenyà, S. Foix and C. Torras. Segmenting color images into surface patches by exploiting sparse depth data, 2011 IEEE Workshop on Applications of Computer Vision, 2011, Kona, Hawaii, pp. 591-598.

E. Serradell, A. Romero, R. Leta, C. Gatta and F. Moreno-Noguer. Simultaneous correspondence and non-rigid 3D reconstruction of the coronary tree from single X-ray images, 13th International Conference on Computer Vision, 2011, Barcelona, pp. 850-857, IEEE.

G. Alenyà, B. Dellen and C. Torras. 3D modelling of leaves from color and ToF data for robotized plant measuring, 2011 IEEE International Conference on Robotics and Automation, 2011, Shanghai, China, pp. 3408-3414, IEEE.

L. Rozo, P. Jiménez and C. Torras. Robot learning from demonstration in the force domain, 2011 IJCAI Workshop on Agents Learning Interactively from Human Teachers, 2011, Barcelona, Spain, pp. 1-6.

J. Sanchez, J. Ostlund, P. Fua and F. Moreno-Noguer. Simultaneous pose, correspondence and non-rigid shape, 2010 IEEE/CVF Conference on Computer Vision and Pattern Recognition, 2010, San Francisco, CA, USA, pp. 1189-1196.

M. Villamizar, F. Moreno-Noguer, J. Andrade-Cetto and A. Sanfeliu. Efficient rotation invariant object detection using boosted random Ferns, 2010 IEEE/CVF Conference on Computer Vision and Pattern Recognition, 2010, San Francisco, CA, USA, pp. 1038-1045.

M. Villamizar, F. Moreno-Noguer, J. Andrade-Cetto and A. Sanfeliu. Shared random Ferns for efficient detection of multiple categories, 20th International Conference on Pattern Recognition, 2010, Istanbul, Turkey, pp. 388-391.

L. Rozo, P. Jiménez and C. Torras. Sharpening haptic inputs for teaching a manipulation skill to a robot, 1st IEEE International Conference on Applied Bionics and Biomechanics, 2010, Venice, pp. 331-340.

F. Moreno-Noguer, J.M. Porta and P. Fua. Exploring ambiguities for monocular non-rigid shape estimation, 11th European Conference on Computer Vision, 2010, Crete, Greece, in Computer Vision - ECCV 2010, Vol 6313 of Lecture Notes in Computer Science, pp. 370-383, 2010, Springer, Berlin.

Other Publications

A. Agostini, C. Torras and F. Wörgötter. A general strategy for interactive decision-making in robotic platforms. Technical Report IRI-TR-11-01, Institut de Robòtica i Informàtica Industrial, CSIC-UPC, 2011.