Research Project
MoHuCo: Modeling Humans in Context
Type
National Project
Start Date
01/09/2021
End Date
31/08/2024
Project Code
PID2020-120049RB-I00
Staff
-
-
Agudo, Antonio
Principal Investigator
-
Sanchez, Jordi
Researcher
-
Pérez, Raül
PhD Student
-
Pérez, Marc
PhD Student
Project Description
Project PID2020-120049RB-I00 funded by MCIN/ AEI /10.13039/501100011033
Recent advances in computer vision and deep learning have shown impressive results in modelling different aspects of humans. Given a single image or a video sequence, these models provide detailed reconstructions of the body shape and clothes, predict future movements and understand human behaviour, emotions and intentions. However, one essential factor that has been obviated so far, is the fact that most of these human characteristics are inherently driven by interactions with objects and/or other people in the environment. For instance, the body trajectory is highly constrained by the spatial distribution of the rest of objects in the environment; a particular facial expression (e.g. ‘fear’) may respond to a specific circumstance occurring in the surrounding (e.g. ‘danger’). Understanding these types of human-context connections would allow going beyond current state-of-the-art and perform robust human reasoning under complex situations such as partial observations (e.g. crowded scenes, heavy occlusions) or indirect observations (predicting human characteristics from contextual clues).
The goal of MoHuCo is therefore to develop novel computer vision tools to discover interrelations between person’s properties and the context. For this purpose, we will split the project in three main blocks:
1) Observing the human: consolidation and pushing to the limits the algorithms for 3D body/cloth reconstruction, motion prediction and behaviour analysis given direct observations of the person.
2) Observing the context: research on novel algorithms to extract heterogeneous information (both geometric and semantic) of the environment;
3) Build joint human-context models: bringing the representations of humans and environment into a single model, allowing to indirectly reason about the human from direct observations of the context.
Project Publications
Conference Publications
-
N. Ugrinovic, A. Pumarola, A. Sanfeliu and F. Moreno-Noguer. Single-view 3d body and cloth reconstruction under complex poses, 17th International Conference on Computer Vision Theory and Applications, 2022, Online.
Abstract
Info
PDF
-
P. Estevez and A. Agudo. Uncalibrated, unified and unsupervised specular-aware photometric stereo, 2022 ICPR Workshop on Towards a Complete Analysis of People: From Face and Body to Clothes, 2022, Montreal (Canada), to appear.
Abstract
Info
PDF
-
G.D. Delmas, P. Weinzaepfel, T. Lucas, F. Moreno-Noguer and G. Rogez. PoseScript: 3D human poses from natural language, 17th European Conference on Computer Vision, 2022, Tel Aviv (Israel), in Computer Vision – ECCV 2022 , Vol 13666 of Lecture Notes in Computer Science, pp. 346-362, 2022.
Abstract
Info
PDF
-
E. Corona, G. Pons-Moll, G. Alenyà and F. Moreno-Noguer. Learned Vertex Descent: a new direction for 3D human model fitting, 17th European Conference on Computer Vision, 2022, Tel Aviv (Israel), in Computer Vision – ECCV 2022 , Vol 13666 of Lecture Notes in Computer Science, pp. 146--165, 2022.
Abstract
Info
PDF
-
J. Shen, A. Agudo, F. Moreno-Noguer and A. Ruiz. Conditional-Flow NeRF: Accurate 3D modelling with reliable uncertainty quantification, 17th European Conference on Computer Vision, 2022, Tel Aviv (Israel), in Computer Vision – ECCV 2022 , Vol 13666 of Lecture Notes in Computer Science, pp. 540-557, 2022.
Abstract
Info
PDF
-
A. Pérez and A. Agudo. Matching and recovering 3D people from multiple views, 2022 IEEE Winter Conference on Applications of Computer Vision, 2022, Waikoloa, Hawaii, USA, pp. 1184-1193, IEEE.
Abstract
Info
PDF
-
A. Agudo. Safari from visual signals: Recovering volumetric 3D shapes, 2022 IEEE International Conference on Acoustics, Speech and Signal Processing, 2022, Singapore, pp. 2495-2499.
Abstract
Info
PDF
-
D.F. Ordoñez, A. Agudo, F. Moreno-Noguer and M. Martin. An adaptable approach to learn realistic legged locomotion without examples, 2022 IEEE International Conference on Robotics and Automation, 2022, Philadelphia, Pennsylvania, USA, pp. 4671-4678.
Abstract
Info
PDF
-
A. Agudo. Spline human motion recovery, 2022 IEEE International Conference on Image Processing, 2022, Bordeaux, France, pp. 4138-4142, IEEE.
Abstract
Info
PDF
-
A. Ruiz, A. Agudo and F. Moreno-Noguer. Generating attribution maps with disentangled masked backpropagation, 2021 International Conference on Computer Vision, 2021, (Virtual), pp. 885-894.
Abstract
Info
PDF
-
J. Sanchez, A. Pumarola and F. Moreno-Noguer. PhysXNet: A customizable approach for learning cloth dynamics on dressed people, 2021 International Conference on 3D Vision, 2021, London, UK (Virtual), pp. 879-888.
Abstract
Info
PDF
-
E. Ramon, G. Triginer, J. Escur, A. Pumarola, J. García, X. Giro-i-Nieto and F. Moreno-Noguer. H3D-Net: Few-shot high-fidelity 3D head reconstruction, 2021 International Conference on Computer Vision, 2021, (Virtual), pp. 5620-5629, to appear.
Abstract
Info
PDF
-
A. Hernandez Ruiz, A. Vilalta and F. Moreno-Noguer. Neural Cellular Automata manifold, 2021 IEEE/CVF Conference on Computer Vision and Pattern Recognition, 2021, Nashville, TN, USA (Virtual), pp. 10015-10023, Computer Vision Foundation.
Abstract
Info
PDF
-
N. Ugrinovic, A. Ruiz, A. Agudo, A. Sanfeliu and F. Moreno-Noguer. Body size and depth disambiguation in multi-person reconstruction from single images, 2021 International Conference on 3D Vision, 2021, London, UK (Virtual), pp. 53-63.
Abstract
Info
PDF
-
J. Shen, A. Ruiz, A. Agudo and F. Moreno-Noguer. Stochastic Neural Radiance Fields: Quantifying uncertainty in implicit 3D representations, 2021 International Conference on 3D Vision, 2021, London, UK (Virtual), pp. 972-981.
Abstract
Info
PDF
-
A. Chatziagapi, S. Athar, F. Moreno-Noguer and D. Samaras. SIDER: Single-image neural optimization for facial geometric detail recovery, 2021 International Conference on 3D Vision, 2021, London, UK (Virtual), pp. 815-824.
Abstract
Info
PDF
Follow us!