IRI - CANOPIES: A Collaborative Paradigm for Human Workers and Multi-Robot Teams in Precision Agriculture Systems

Research Project

CANOPIES: A Collaborative Paradigm for Human Workers and Multi-Robot Teams in Precision Agriculture Systems

Type

European Project

Start Date

01/01/2021

End Date

31/12/2024

Project Code

H2020-ICT-2020-2-101016906

Staff

Sanfeliu, Alberto

Principal Investigator
del Pino, Iván

Researcher

Alquézar, Renato

Researcher
Andrade, Juan

Researcher

Puig-Pey, Ana Maria

Researcher
Garrell, Anaís

Researcher

Dalmasso, Marc

PhD Student
Domínguez, José Enrique

PhD Student

Herrero, Fernando

Support
Bolea, Yolanda

Member

Martí, Josep

Member
Laplaza, Javier

Member

Moreno, Francesc

Member
Guerra, Edmundo

Member

Grau, Antoni

Member

Project Description

This project has received funding from the European Union’s Horizon 2020 research and innovation programme under grant agreement No 101016906

In CANOPIES, our goal is to develop a novel collaborative human-robot paradigm addressing the challenges of Human Robot Interaction and Human-Robot Collaboration in the unstructured highly dynamic outdoor environment of permanent crop farming (Agri-Food Area). Our approach will be demonstrated through an integrated system composed by farming robots and logistics robots with a real-world validation of two economically relevant agronomic operations within a table-grape vineyard: harvesting and pruning. CANOPIES represents the first attempt to introduce a collaborative paradigm in the field of precision agriculture for permanent crops where farmworkers can efficiently work together with teams of robots to perform agronomic interventions, like harvesting or pruning in table-grape vineyards. Both operations require complex processes of perception, communication, shared planning in agreement, prediction of human intentions, interaction and action. But also, both agronomic operations should be done in real life conditions, that is, in changing illumination and cast shadows, changing agronomic situations, where the vine branches or grapes can make it difficult to harvest or prune in a safe manner, due to the robot physical proximity to the human, while operating in real time. CANOPIES ambition will be achieved by introducing: i) novel human-robot interaction methodologies for enhanced safety and coexistence, ii) novel human-robot collaboration methodologies for increased system adaptability and intuitive usability; iii) novel multi-robot coordination methodologies for improved scalability. CANOPIES impact will contribute to filling the current gap in the development of fully autonomous robotic solutions for permanent crops by introducing a novel concept of farming robots, where we leverage an effective interaction with the human workers to mitigate the greater complexity of permanent crops as compared with field crops.

Website: http://www.canopies-project.eu/

Project Publications

Journal Publications

O. Gil and A. Sanfeliu. Human-robot collaborative minimum time search through sub-priors in ant colony optimization. IEEE Robotics and Automation Letters, 9(11): 10216-10223, 2024.

Abstract Info PDF
J. Laplaza, F. Moreno-Noguer and A. Sanfeliu. Enhancing robotic collaborative tasks through contextual human motion prediction and intention inference. International Journal of Social Robotics: 1-20, 2024, to appear.

Abstract Info PDF
P.T. Singamaneni, P. Bachiller, L.J. Manso, A. Garrell Zulueta, A. Sanfeliu, A. Spalanzani and R. Alami. A survey on socially aware robot navigation: taxonomy and future challenges. The International Journal of Robotics Research, 43(10): 1533-1572, 2024.

Abstract Info PDF
J.E. Domínguez, N.A. Rodríguez and A. Sanfeliu. Perception–intention–action cycle in human–robot collaborative tasks: The collaborative lightweight object transportation use-case. International Journal of Social Robotics, 2024, to appear.

Abstract Info PDF
T. Ciarfuglia, I. Motoi, L. Saraceni, M. Fawakherji, A. Sanfeliu and D. Nardi. Weakly and semi-supervised detection, segmentation and tracking of table grapes with limited and noisy data. Computers and Electronics in Agriculture, 205: 107624, 2023.

Abstract Info PDF
G. Coll, I.J. Torres, A. Grau, E. Guerra and A. Sanfeliu. Accurate detection and depth estimation of table grapes and peduncles for robot harvesting, combining monocular depth estimation and CNN methods. Computers and Electronics in Agriculture: 108362, 2023.

Abstract Info PDF
M. Saboia Da Silva, L. Clark, V. Thangavelu, J. Edlund, K. Otsu, G.J. Correa, V.S. Varadharajan, A. Santamaria-Navarro, T. Touma, A. Bouman, H. Melikyan, T. Pailevanian, S. Kim, A. Archanian, T.S. Vaquero, G. Beltrame, N. Napp, G. Pessin and A. Agha-mohammadi. ACHORD: Communication-aware multi-robot coordination with intermittent connectivity. IEEE Robotics and Automation Letters, 7(4): 10184-10191, 2022.

Abstract Info PDF
W.O. Chamorro, J. Solà and J. Andrade-Cetto. Event-based line SLAM in real-time. IEEE Robotics and Automation Letters, 7(3): 8146-8153, 2022.

Abstract Info PDF
M. Peral, A. Sanfeliu and A. Garrell Zulueta. Efficient hand gesture recognition for human-robot interaction. IEEE Robotics and Automation Letters, 7(4): 10272-10279, 2022.

Abstract Info PDF
O. Gil, A. Garrell Zulueta and A. Sanfeliu. Social robot navigation tasks: Combining machine learning techniques and Social Force Model. Sensors, 21(7087): 23, 2021.

Abstract Info PDF

Conference Publications

J.E. Domínguez and A. Sanfeliu. Exploring transformers and visual transformers for force prediction in human-robot collaborative transportation tasks, 2024 IEEE International Conference on Robotics and Automation, 2024, Yokohama (Japan), pp. 3191-3197.

Abstract Info PDF
J.E. Domínguez and A. Sanfeliu. Anticipation and proactivity. Unraveling both concepts in human-robot interaction through a handover example, 33rd IEEE International Symposium on Robot and Human Interactive Communication, 2024, Pasadena, California, USA, pp. 957-962, IEEE.

Abstract Info PDF
J.E. Domínguez and A. Sanfeliu. Force and velocity prediction in human-robot collaborative transportation tasks through video retentive networks, 2024 IEEE/RSJ International Conference on Intelligent Robots and Systems, 2024, Abu Dhabi, UAE, pp. 9307-9313.

Abstract Info PDF
M. Dalmasso, F. Garcia-Ruiz, T. Ciarfuglia, L. Saraceni, I. M. Motoi, E. Gil and A. Sanfeliu. Human-robot harvesting plan negotiation: Perception, grape mapping and shared planning, 7th Iberian Robotics Conference, 2024, Madrid, Spain, pp. 1-8.

Abstract Info PDF
J. Laplaza, J.J. Oliver, A. Sanfeliu and A. Garrell Zulueta. Body gestures recognition for social human-robot interaction, 7th Iberian Robotics Conference, 2024, Madrid, Spain.

Abstract Info PDF
J. Palmieri, P. Di Lillo, A. Sanfeliu and A. Marino. Perception-driven shared control architecture for agricultural robots performing harvesting tasks, 2024 IEEE/RSJ International Conference on Intelligent Robots and Systems, 2024, Abu Dhabi, UAE, pp. 9328-9334.

Abstract Info PDF
P. Falqueto, A. Sanfeliu, L. Palopoli and D. Fontanelli. Learning priors of human motion with vision transformers, 48th IEEE International Conference on Computers, Software, and Applications , 2024, Osaka, Japan, pp. 382-389.

Abstract Info PDF
J.E. Domínguez and A. Sanfeliu. Voice Command Recognition for Explicit Intent Elicitation in Collaborative Object Transportation Tasks: a ROS-based Implementation, 2024 ACM/IEEE International Conference on Human-Robot Interaction, 2024, Boulder, CO, USA, pp. 412–416.

Abstract Info PDF
M. Dalmasso, V. Sanchez-Anguix, A. Garrell Zulueta, P. Jiménez and A. Sanfeliu. Exploring Preferences in Human-Robot Navigation Plan Proposal Representation, 2024 ACM/IEEE International Conference on Human-Robot Interaction, 2024, Boulder, CO, USA, pp. 369-373.

Abstract Info PDF
J. Laplaza, R. Romero, A. Sanfeliu and A. Garrell Zulueta. Body gesture recognition to control a social mobile robot, 2023 ACM/IEEE International Conference on Human-Robot Interaction, 2023, Stockholm, in Companion of the HRI'23, pp. 456-460.

Abstract Info PDF
J.E. Domínguez, N.A. Rodríguez and A. Sanfeliu. Perception-intention-action cycle as a human acceptable way for improving human-robot collaborative tasks, 2023 ACM/IEEE International Conference on Human-Robot Interaction, 2023, Stockholm, in Companion of the HRI'23, pp. 567-571.

Abstract Info PDF
E. Repiso, A. Garrell Zulueta and A. Sanfeliu. Real-life experiment metrics for evaluating human-robot collaborative navigation tasks, 32nd IEEE International Symposium on Robot and Human Interactive Communication, 2023, Busan, Korea, pp. 660-667.

Abstract Info PDF
O. Gil and A. Sanfeliu. Human motion trajectory prediction using the Social Force Model for real-time and low computational cost applications, 6th Iberian Robotics Conference, 2023, Coimbra, Portugal, pp. 235–247, Springer.

Abstract Info PDF
J.E. Domínguez and A. Sanfeliu. Inference vs. explicitness. Do we really need the perfect predictor? The human-robot collaborative object transportation case, 32nd IEEE International Symposium on Robot and Human Interactive Communication, 2023, Busan, Korea, pp. 1866-1871.

Abstract Info PDF
J.E. Domínguez and A. Sanfeliu. Improving human-robot interaction effectiveness in human-robot collaborative object transportation using force prediction, 2023 IEEE/RSJ International Conference on Intelligent Robots and Systems, 2023, Detroit, MI, USA, pp. 7839-7845.

Abstract Info PDF
O. Castro, E. Repiso, A. Garrell Zulueta and A. Sanfeliu. Classification of humans social relations within urban areas, 5th Iberian Robotics Conference, 2022, Zaragoza, Spain, Vol 590 of Lecture Notes in Networks and Systems, pp. 27-39, 2023.

Abstract Info PDF
J. Laplaza, F. Moreno-Noguer and A. Sanfeliu. Context attention: Human motion prediction using context information and deep learning attention models, 5th Iberian Robotics Conference, 2022, Zaragoza, Spain, Vol 590 of Lecture Notes in Networks and Systems, pp. 102-112, 2023.

Abstract Info PDF
J. Laplaza, N.A. Rodríguez, J.E. Domínguez, F. Herrero, S. Hernández, A. López, A. Sanfeliu and A. Garrell Zulueta. IVO Robot: A new social robot for Human-Robot collaboration, 2022 ACM/IEEE International Conference on Human-Robot Interaction, 2022, Sapporo, Japan, pp. 860–864.

Abstract Info PDF
J. Laplaza, F. Moreno-Noguer and A. Sanfeliu. Context and intention aware 3D human body motion prediction using an attention deep learning model in handover tasks, 2022 IEEE/RSJ International Conference on Intelligent Robots and Systems, 2022, Kyoto, pp. 4743-4748.

Abstract Info PDF
J. Laplaza, A. Garrell Zulueta, F. Moreno-Noguer and A. Sanfeliu. Context and intention for 3D human motion prediction: Experimentation and user study in handover tasks, 31st IEEE International Symposium on Robot and Human Interactive Communication, 2022, Napoli, Italy, pp. 630-635.

Abstract Info PDF
J. Laplaza, A. Pumarola, F. Moreno-Noguer and A. Sanfeliu. Attention deep learning based model for predicting the 3D human body pose using the robot human handover phases, 30th IEEE International Symposium on Robot and Human Interactive Communication, 2021, Vancouver, Canada, pp. 161-166.

Abstract Info PDF

Institut de Robòtica i Informàtica Industrial, CSIC-UPC
C/ Llorens i Artigas 4-6, 08028, Barcelona, Spain

Site map
Accessibility
About this web & cookies
Disclaimer

The activities of our institute are supported by:

Research Project

CANOPIES: A Collaborative Paradigm for Human Workers and Multi-Robot Teams in Precision Agriculture Systems

Type

Start Date

End Date

Project Code

Staff

Principal Investigator

Researcher

Researcher

Researcher

Researcher

Researcher

PhD Student

PhD Student

Support

Member

Member

Member

Member

Member

Member

Project Description

Project Publications

Journal Publications

O. Gil and A. Sanfeliu. Human-robot collaborative minimum time search through sub-priors in ant colony optimization. IEEE Robotics and Automation Letters, 9(11): 10216-10223, 2024.

J. Laplaza, F. Moreno-Noguer and A. Sanfeliu. Enhancing robotic collaborative tasks through contextual human motion prediction and intention inference. International Journal of Social Robotics: 1-20, 2024, to appear.

P.T. Singamaneni, P. Bachiller, L.J. Manso, A. Garrell Zulueta, A. Sanfeliu, A. Spalanzani and R. Alami. A survey on socially aware robot navigation: taxonomy and future challenges. The International Journal of Robotics Research, 43(10): 1533-1572, 2024.

J.E. Domínguez, N.A. Rodríguez and A. Sanfeliu. Perception–intention–action cycle in human–robot collaborative tasks: The collaborative lightweight object transportation use-case. International Journal of Social Robotics, 2024, to appear.

T. Ciarfuglia, I. Motoi, L. Saraceni, M. Fawakherji, A. Sanfeliu and D. Nardi. Weakly and semi-supervised detection, segmentation and tracking of table grapes with limited and noisy data. Computers and Electronics in Agriculture, 205: 107624, 2023.

G. Coll, I.J. Torres, A. Grau, E. Guerra and A. Sanfeliu. Accurate detection and depth estimation of table grapes and peduncles for robot harvesting, combining monocular depth estimation and CNN methods. Computers and Electronics in Agriculture: 108362, 2023.

W.O. Chamorro, J. Solà and J. Andrade-Cetto. Event-based line SLAM in real-time. IEEE Robotics and Automation Letters, 7(3): 8146-8153, 2022.

M. Peral, A. Sanfeliu and A. Garrell Zulueta. Efficient hand gesture recognition for human-robot interaction. IEEE Robotics and Automation Letters, 7(4): 10272-10279, 2022.

O. Gil, A. Garrell Zulueta and A. Sanfeliu. Social robot navigation tasks: Combining machine learning techniques and Social Force Model. Sensors, 21(7087): 23, 2021.

Conference Publications

J.E. Domínguez and A. Sanfeliu. Exploring transformers and visual transformers for force prediction in human-robot collaborative transportation tasks, 2024 IEEE International Conference on Robotics and Automation, 2024, Yokohama (Japan), pp. 3191-3197.

J.E. Domínguez and A. Sanfeliu. Anticipation and proactivity. Unraveling both concepts in human-robot interaction through a handover example, 33rd IEEE International Symposium on Robot and Human Interactive Communication, 2024, Pasadena, California, USA, pp. 957-962, IEEE.

J.E. Domínguez and A. Sanfeliu. Force and velocity prediction in human-robot collaborative transportation tasks through video retentive networks, 2024 IEEE/RSJ International Conference on Intelligent Robots and Systems, 2024, Abu Dhabi, UAE, pp. 9307-9313.

M. Dalmasso, F. Garcia-Ruiz, T. Ciarfuglia, L. Saraceni, I. M. Motoi, E. Gil and A. Sanfeliu. Human-robot harvesting plan negotiation: Perception, grape mapping and shared planning, 7th Iberian Robotics Conference, 2024, Madrid, Spain, pp. 1-8.

J. Laplaza, J.J. Oliver, A. Sanfeliu and A. Garrell Zulueta. Body gestures recognition for social human-robot interaction, 7th Iberian Robotics Conference, 2024, Madrid, Spain.

J. Palmieri, P. Di Lillo, A. Sanfeliu and A. Marino. Perception-driven shared control architecture for agricultural robots performing harvesting tasks, 2024 IEEE/RSJ International Conference on Intelligent Robots and Systems, 2024, Abu Dhabi, UAE, pp. 9328-9334.

P. Falqueto, A. Sanfeliu, L. Palopoli and D. Fontanelli. Learning priors of human motion with vision transformers, 48th IEEE International Conference on Computers, Software, and Applications , 2024, Osaka, Japan, pp. 382-389.

J.E. Domínguez and A. Sanfeliu. Voice Command Recognition for Explicit Intent Elicitation in Collaborative Object Transportation Tasks: a ROS-based Implementation, 2024 ACM/IEEE International Conference on Human-Robot Interaction, 2024, Boulder, CO, USA, pp. 412–416.

M. Dalmasso, V. Sanchez-Anguix, A. Garrell Zulueta, P. Jiménez and A. Sanfeliu. Exploring Preferences in Human-Robot Navigation Plan Proposal Representation, 2024 ACM/IEEE International Conference on Human-Robot Interaction, 2024, Boulder, CO, USA, pp. 369-373.

J. Laplaza, R. Romero, A. Sanfeliu and A. Garrell Zulueta. Body gesture recognition to control a social mobile robot, 2023 ACM/IEEE International Conference on Human-Robot Interaction, 2023, Stockholm, in Companion of the HRI'23, pp. 456-460.

J.E. Domínguez, N.A. Rodríguez and A. Sanfeliu. Perception-intention-action cycle as a human acceptable way for improving human-robot collaborative tasks, 2023 ACM/IEEE International Conference on Human-Robot Interaction, 2023, Stockholm, in Companion of the HRI'23, pp. 567-571.

E. Repiso, A. Garrell Zulueta and A. Sanfeliu. Real-life experiment metrics for evaluating human-robot collaborative navigation tasks, 32nd IEEE International Symposium on Robot and Human Interactive Communication, 2023, Busan, Korea, pp. 660-667.

O. Gil and A. Sanfeliu. Human motion trajectory prediction using the Social Force Model for real-time and low computational cost applications, 6th Iberian Robotics Conference, 2023, Coimbra, Portugal, pp. 235–247, Springer.

J.E. Domínguez and A. Sanfeliu. Inference vs. explicitness. Do we really need the perfect predictor? The human-robot collaborative object transportation case, 32nd IEEE International Symposium on Robot and Human Interactive Communication, 2023, Busan, Korea, pp. 1866-1871.

J.E. Domínguez and A. Sanfeliu. Improving human-robot interaction effectiveness in human-robot collaborative object transportation using force prediction, 2023 IEEE/RSJ International Conference on Intelligent Robots and Systems, 2023, Detroit, MI, USA, pp. 7839-7845.

O. Castro, E. Repiso, A. Garrell Zulueta and A. Sanfeliu. Classification of humans social relations within urban areas, 5th Iberian Robotics Conference, 2022, Zaragoza, Spain, Vol 590 of Lecture Notes in Networks and Systems, pp. 27-39, 2023.

J. Laplaza, F. Moreno-Noguer and A. Sanfeliu. Context attention: Human motion prediction using context information and deep learning attention models, 5th Iberian Robotics Conference, 2022, Zaragoza, Spain, Vol 590 of Lecture Notes in Networks and Systems, pp. 102-112, 2023.

J. Laplaza, N.A. Rodríguez, J.E. Domínguez, F. Herrero, S. Hernández, A. López, A. Sanfeliu and A. Garrell Zulueta. IVO Robot: A new social robot for Human-Robot collaboration, 2022 ACM/IEEE International Conference on Human-Robot Interaction, 2022, Sapporo, Japan, pp. 860–864.

J. Laplaza, F. Moreno-Noguer and A. Sanfeliu. Context and intention aware 3D human body motion prediction using an attention deep learning model in handover tasks, 2022 IEEE/RSJ International Conference on Intelligent Robots and Systems, 2022, Kyoto, pp. 4743-4748.

J. Laplaza, A. Garrell Zulueta, F. Moreno-Noguer and A. Sanfeliu. Context and intention for 3D human motion prediction: Experimentation and user study in handover tasks, 31st IEEE International Symposium on Robot and Human Interactive Communication, 2022, Napoli, Italy, pp. 630-635.

J. Laplaza, A. Pumarola, F. Moreno-Noguer and A. Sanfeliu. Attention deep learning based model for predicting the 3D human body pose using the robot human handover phases, 30th IEEE International Symposium on Robot and Human Interactive Communication, 2021, Vancouver, Canada, pp. 161-166.