Shape basis interpretation for monocular deformable 3-D reconstruction

Journal Article (2019)


IEEE Transactions on Multimedia







Doc link


Download the digital copy of the doc pdf document


In this paper, we propose a novel interpretable shape model to encode object non-rigidity. We first use the initial frames of a monocular video to recover a rest shape, used later to compute a dissimilarity measure based on a distance matrix measurement. Spectral analysis is then applied to this matrix to obtain a reduced shape basis, that in contrast to existing approaches, can be physically interpreted. In turn, these pre-computed shape bases are used to linearly span the deformation of a wide variety of objects. We introduce the low-rank basis into a sequential approach to recover both camera motion and non-rigid shape from the monocular video, by simply optimizing the weights of the linear combination using bundle adjustment. Since the number of parameters to optimize per frame is relatively small, specially when physical priors are considered, our approach is fast and can potentially run in real time. Validation is done in a wide variety of real-world objects, undergoing both inextensible and extensible deformations. Our approach achieves remarkable robustness to artifacts such as noisy and missing measurements and shows an improved performance to competing methods.


computer vision.

Author keywords

Deformable Shape Analysis, Dynamic Modeling, Structure from Motion, Low-Rank Representation, Optimization

Scientific reference

A. Agudo and F. Moreno-Noguer. Shape basis interpretation for monocular deformable 3-D reconstruction. IEEE Transactions on Multimedia, 21(4): 821-834, 2019.