Publication
Single-view 3d body and cloth reconstruction under complex poses
Conference Article
Conference
International Conference on Computer Vision Theory and Applications (VISAPP)
Edition
17th
Doc link
https://doi.org/10.5220/0010896100003124
File
Authors
Projects associated
Abstract
Recent advances in 3D human shape reconstruction from single images have shown impressive results, leveraging on deep networks that model the so-called implicit function to learn the occupancy status of arbitrarily dense 3D points in space. However, while current algorithms based on this paradigm, like PiFuHD, are able to estimate accurate geometry of the human shape and clothes, they require high-resolution input images and are not able to capture complex body poses. Most training and evaluation is performed on 1k-resolution images of humans standing in front of the camera under neutral body poses. In this paper, we leverage publicly available data to extend existing implicit function-based models to deal with images of humans that can have arbitrary poses and self-occluded limbs. We argue that the representation power of the implicit function is not sufficient to simultaneously model details of the geometry and of the body pose. We, therefore, propose a coarse-to-fine approach in which we first learn an implicit function that maps the input image to a 3D body shape with a low level of detail, but which correctly fits the underlying human pose, despite its complexity. We then learn a displacement map, conditioned on the smoothed surface and on the input image, which encodes the high-frequency details of the clothes and body. In the experimental section, we show that this coarse-to-fine strategy represents a very good trade-off between shape detail and pose correctness, comparing favorably to the most recent state-of-the-art approaches. Our code will be made publicly available.
Categories
computer vision.
Author keywords
Computer Vision, deep learning, human avatar
Scientific reference
N. Ugrinovic, A. Pumarola, A. Sanfeliu and F. Moreno-Noguer. Single-view 3d body and cloth reconstruction under complex poses, 17th International Conference on Computer Vision Theory and Applications, 2022, Online.
Follow us!