PhD Thesis

Building embedded systems for 3D reconstruction of deformable surfaces in non-invasive surgery applications

Work default illustration


  • If you are interested in the proposal, please contact with the supervisors.


Recent advances in computer vision have shown that it is possible to estimate the 3D shape of deformable surfaces from the sole input of monocular video sequences [1,2,3,4,5]. One of the fields that can mostly benefit from these advances is medical imaging, and in particular the endoscopy surgery, in such a way that the surgeon can have a clear intuition of the internal 3D structure of the body with minimal invasion. However, current computer vision algorithms are not yet ready to be fully exploited in realistic domains. Some of the issues that need to tackled are: (1) robustness to non-textured surfaces, hard illumination artifacts and severe occlusions; (2) real-time computation at frame rate; (3) execution on light, low-consuming and portable devices. In this thesis, we will seek to overcome these issues, and will develop new embedded devices, specially dedicated to simultaneously recover non-rigid shape and camera pose in endoscopic surgery. As a first main objective we will customize recent algorithms (e.g. [1]) to work on the specific environments we encounter in this kind of surgery. Camera motion models, shape and texture priors will be learned for improving the efficiency and robustness of existing methods, so far, built from a generic perspective. Once we have optimized our algorithms for this specific scenario we will embed them integrate them in embedded devices, with low consuming and small cameras that can be mounted on either rigid or flexible endoscopes. This thesis will be carried on in the Institut de Robòtica i Informàtica Industrial, at the Universitat Politècnica de Catalunya (under direction of Dr. Francesc Moreno and Dr. Antonio Agudo), and with the collaboration of the Hospital Clinic, in Barcelona.

[1] A. Agudo, F. Moreno-Noguer, B. Calvo and J.M.M. Montiel. Sequential Non-Rigid Structure from Motion using Physical Priors. In PAMI, 2016.

[2] A. Agudo and F. Moreno-Noguer. Simultaneous pose and non-rigid shape with particle dynamics. In CVPR, 2015.

[3] R. Garg, A. Roussos, and L. Agapito. Dense variational reconstruction of non-rigid surfaces from monocular video. In CVPR, 2013.

[4] Y. Dai, H. Li and M. He. A simple prior-free method for non-rigid structure from motion factorization. In CVPR, 2012.

[5] M. Lee, J. Cho, C. H. Choi and S. Oh. Procrustean normal distribution for non-rigid structure from motion. In CVPR, 2013.

The work is under the scope of the following projects:

  • RobInstruct: Instructing robots using natural communication skills (web)