Publication

Total estimation from RGB video: On-line camera self-calibration, non-rigid shape and motion

Conference Article

Conference

International Conference on Pattern Recognition (ICPR)

Edition

25th

File

Download the digital copy of the doc pdf document

Abstract

In this paper we present a sequential approach to jointly retrieve camera auto-calibration, camera pose and the 3D reconstruction of a non-rigid object from an uncalibrated RGB image sequence, without assuming any prior information about the shape structure, nor the need for a calibration pattern, nor the use of training data at all. To this end, we propose a Bayesian filtering approach based on a sum-of-Gaussians filter composed of a bank of extended Kalman filters (EKF). For every EKF, we make use of dynamic models to estimate its state vector, which later will be Gaussianly combined to achieve a global solution. To deal with deformable objects, we incorporate a mechanical model solved by using the finite element method. Thanks to these ingredients, the resulting method is both efficient and robust to several artifacts such as missing and noisy observations as well as sudden camera motions, while being available for a wide variety of objects and materials, including isometric and elastic shape deformations. Experimental validation is proposed in real experiments, showing its strengths with respect to competing approaches.

Categories

computer vision, optimisation.

Author keywords

Self-Calibration, Non-Rigid Shape and Motion, Filtering

Scientific reference

A. Agudo. Total estimation from RGB video: On-line camera self-calibration, non-rigid shape and motion, 25th International Conference on Pattern Recognition, 2021, Online, to appear.