Publication

Sequential non-rigid structure from motion using physical priors

Journal Article (2016)

Journal

IEEE Transactions on Pattern Analysis and Machine Intelligence

Pages

979-994

Volume

38

Number

5

Doc link

http://dx.doi.org/10.1109/TPAMI.2015.2469293

File

Download the digital copy of the doc pdf document

Abstract

We propose a new approach to simultaneously recover camera pose and 3D shape of non-rigid and potentially extensible surfaces from a monocular image sequence. For this purpose, we make use of the EKF-SLAM (Extended Kalman Filter based Simultaneous Localization And Mapping) formulation, a Bayesian optimization framework traditionally used in mobile robotics for estimating camera pose and reconstructing rigid scenarios. In order to extend the problem to a deformable domain we represent the object’s surface mechanics by means of Navier’s equations, which are solved using a FEM (Finite Element Method). With these main ingredients, we can further model the material’s stretching, allowing us to go a step further than most of current techniques, typically constrained to surfaces undergoing isometric deformations. We extensively validate our approach in both real and synthetic experiments, and demonstrate its advantages with respect to competing methods. More specifically, we show that besides simultaneously retrieving camera pose and non-rigid shape, our approach is adequate for both isometric and extensible surfaces, does not require neither batch processing all the frames nor tracking points over the whole sequence and runs at several frames per second.

Categories

computer vision.

Scientific reference

A. Agudo, F. Moreno-Noguer, B. Calvo and J.M. Martínez. Sequential non-rigid structure from motion using physical priors. IEEE Transactions on Pattern Analysis and Machine Intelligence, 38(5): 979-994, 2016.