Publication
Robust spatio-temporal clustering and reconstruction of multiple deformable bodies
Journal Article (2019)
Journal
IEEE Transactions on Pattern Analysis and Machine Intelligence
Pages
971-984
Volume
41
Number
4
Doc link
https://doi.org/10.1109/TPAMI.2018.2823717
File
Abstract
In this paper we present an approach to reconstruct the 3D shape of multiple deforming objects from a collection of sparse, noisy and possibly incomplete 2D point tracks acquired by a single monocular camera. Additionally, the proposed solution estimates the camera motion and reasons about the spatial segmentation (i.e., identifies each of the deforming objects in every frame) and temporal clustering (i.e., splits the sequence into motion primitive actions). This advances competing work, which mainly tackled the problem for one single object and non-occluded tracks. In order to handle several objects at a time from partial observations, we model point trajectories as a union of spatial and temporal subspaces, and optimize the parameters of both modalities, the non-observed point tracks, the camera motion, and the time-varying 3D shape via augmented Lagrange multipliers. The algorithm is fully unsupervised and does not require any training data at all. We thoroughly validate the method on challenging scenarios with several human subjects performing different activities which involve complex motions and close interaction. We show our approach achieves state-of-the-art 3D reconstruction results, while it also provides spatial and temporal segmentation.
Categories
computer vision, optimisation.
Author keywords
Non-Rigid Structure from Motion, Union of Subspaces, Spatio-Temporal Clustering, Augmented Lagrange Multipliers
Scientific reference
A. Agudo and F. Moreno-Noguer. Robust spatio-temporal clustering and reconstruction of multiple deformable bodies. IEEE Transactions on Pattern Analysis and Machine Intelligence, 41(4): 971-984, 2019.
Follow us!