Perception and Action in Robotics

National DPI Projects PAU (2009-2011) and PAU+ (2012-2014)

Pose and shape recovery of rigid and non-rigid objects

Complete list of publications

Highlighted Publications

Soft segmentation cues: We show as RGB maps the first three coordinates of the ‘Eigen’ embeddings and the ‘SoftMask’ embeddings
Dense segmentation-aware descriptors.
E. Trulls, I. Kokkinos, A. Sanfeliu, F. Moreno-Noguer.
Conference on Computer Vision and Pattern Recognition (CVPR), 2013 (to appear)
Show Abstract

In this work we exploit segmentation to construct appearance descriptors that can robustly deal with occlusion and background changes. For this, we downplay measure- ments coming from areas that are unlikely to belong to the same region as the descriptor’s center, as suggested by soft segmentation masks. Our treatment is applicable to any im- age point, i.e. dense, and its computational overhead is in the order of a few seconds.

Simultaneous estimation of 2D and 3D pose
A joint model for 2D and 3D pose estimation from a single image
E. Simo-Serra, A. Quattoni, C. Torras and F. Moreno-Noguer.
2013 IEEE Computer Society Conference on Computer Vision and Pattern Recognition, 2013, Portland, USA, pp. 3634--3641.
Show Abstract

We introduce a novel approach to automatically recover 3D human pose from a single image. We address this issue by jointly solving both the 2D detection and the 3D inference problems. For this purpose, we propose a Bayesian framework that integrates a generative model based on latent variables and discriminative 2D part detectors based on HOGs, and perform inference using evolutionary algorithms.

Reconstruction results for the two real sequences.
Stochastic exploration of ambiguities for nonrigid shape recovery..
F. Moreno-Noguer and P. Fua.
IEEE Transactions on Pattern Analysis and Machine Intelligence, 35(2): 463-475, 2013.
Show Abstract

We introduce a stochastic sampling approach to efficiently explore the set of solutions of an objective function based on point correspondences. This allows us to propose a small set of ambiguous candidate 3D shapes and then use additional image information to choose the best one. As a proof of concept, we use either motion or shading cues to this end and show that we can handle a complex objective function without having to solve a difficult nonlinear minimization problem.

Search for correspondences in the retinal fundus images
Robust non-rigid registration of 2D and 3D graphs.
E. Serradell, P. Glowacki, K. Jan, F. Moreno-Noguer and P. Fua.
2012 IEEE Computer Society Conference on Computer Vision and Pattern Recognition, 2012, Providence, Rhode Island, pp. 996-1003, IEEE Press.
Show Abstract

We present a new approach to matching graphs embedded in R2 or R3. To handle arbitrary non-linear deformations, we represent them as Gaussian Processes. In the absence of appearance information, we iteratively establish correspondences between graph nodes, update the structure accordingly, and use the current mapping estimate to find the most likely correspondences that will be used in the next iteration.

Results on a real sequence with increasing focal length
Exhaustive linearization for robust camera pose and focal length estimation.
A. Penate-Sanchez, J. Andrade-Cetto and F. Moreno-Noguer.
IEEE Transactions on Pattern Analysis and Machine Intelligence, 35(10): 2387-2400, 2013.
Show Abstract

We propose a novel approach for the estimation of the pose and focal length of a camera from a set of 3D-to-2D point correspondences. Our method compares favorably to competing approaches in that it is both more accurate than existing closed form solutions, as well as faster and also more accurate than iterative ones. We present new methodologies to circumvent this limitation termed exhaustive linearization and exhaustive relinearization which perform a systematic exploration of the solution space in closed form.