Publication
Unsupervised person image synthesis in arbitrary poses
Conference Article
Conference
IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR)
Edition
2018
Pages
8620-8628
Doc link
https://doi.org/10.1109/CVPR.2018.00899
File
Authors
Projects associated
AEROARMS: AErial RObotics System integrating multiple ARMS and advanced manipulation capabilities for inspection and maintenance
ColRobTransp: Colaboración robots-humanos para el transporte de productos en zonas urbanas
Google Faculty Award: Geometry-aware CNNs for Non-Rigid Shape Reconstruction
MdM: Unit of Excellence María de Maeztu
HuMoUR: Markerless 3D human motion understanding for adaptive robot behavior
Abstract
We present a novel approach for synthesizing photo-realistic images of people in arbitrary poses using generative adversarial learning. Given an input image of a person and a desired pose represented by a 2D skeleton, our model renders the image of the same person under the new pose, synthesizing novel views of the parts visible in the input image and hallucinating those that are not seen. This problem has recently been addressed in a supervised manner, i.e., during training the ground truth images under the new poses are given to the network. We go beyond these approaches by proposing a fully unsupervised strategy. We tackle this challenging scenario by splitting the problem into two principal subtasks. First, we consider a pose conditioned bidirectional generator that maps back the initially rendered image to the original pose, hence being directly comparable to the input image without the need to resort to any training image. Second, we devise a novel loss function that incorporates content and style terms, and aims at producing images of high perceptual quality. Extensive experiments conducted on the DeepFashion dataset demonstrate that the images rendered by our model are very close in appearance to those obtained by fully supervised approaches.
Categories
computer vision, optimisation.
Author keywords
GANs; Deep Learning; Conditioned Image Generation
Scientific reference
A. Pumarola, A. Agudo, A. Sanfeliu and F. Moreno-Noguer. Unsupervised person image synthesis in arbitrary poses, 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition, 2018, Salt Lake City, UT, USA, pp. 8620-8628.
Follow us!