Publication

SIRA: Relightable Avatars from a Single Image

Conference Article

Conference

IEEE Winter Conference on Applications of Computer Vision (WACV)

Edition

2023

Pages

775-784

Doc link

https://doi.org/10.1109/WACV56688.2023.00084

File

Download the digital copy of the doc pdf document

Authors

Projects associated

Abstract

Recovering the geometry of a human head from a single image, while factorizing the materials and illumination, is a severely ill-posed problem that requires prior information to be solved. Methods based on 3D Morphable Models (3DMM), and their combination with differentiable renderers, have shown promising results. However, the expressiveness of 3DMMs is limited, and they typically yield oversmoothed and identity-agnostic 3D shapes limited to the face region. Highly accurate full head reconstructions have recently been obtained with neural fields that parameterize the geometry using multilayer perceptrons. The versatility of these representations has also proved effective for disentangling geometry, materials and lighting. However, these methods require several tens of input images. In this paper, we introduce SIRA, a method which, from a single image, reconstructs human head avatars with high fidelity geometry and factorized lights and surface materials. Our key ingredients are two data-driven statistical models based on neural fields that resolve the ambiguities of single-view 3D surface reconstruction and appearance factorization. Experiments show that SIRA obtains state of the art results in 3D head reconstruction while at the same time it successfully disentangles the global illumination, and the diffuse and specular albedos. Furthermore, our reconstructions are amenable to physically-based appearance editing and head model relighting.

Categories

pattern recognition.

Author keywords

Computer Vision, 3D reconstruction, Implicit functions, Scene Factorization

Scientific reference

P. Caselles, E. Ramon, J. Garcia, X. Giro-i-Nieto, F. Moreno-Noguer and G. Triginer. SIRA: Relightable Avatars from a Single Image, 2023 IEEE Winter Conference on Applications of Computer Vision, 2023, Waikoloa, Hawaii, pp. 775-784.