Estimación de flujo óptico en sistemas con luz y cámara solidarios para endoscopia monocular.

Royo Meneses, Diego
Martínez Montiel, José María (dir.)

Universidad de Zaragoza, EINA, 2022
Departamento de Informática e Ingeniería de Sistemas, Área de Ingeniería de Sistemas y Automática

Máster Universitario en Robótica, Gráficos y Visión por Computador

Resumen: The European EndoMapper project aims to develop real-time mapping of the interior of the human body using only footage from an exploration procedure. This technology can enable novel medical operations that include robotized autonomous interaction and live augmented reality. All of this comes down to two principles that need to be overcome: first, generating a map of the human body, and then, being able to locate oneself within it. Simultaneous Localization And Mapping (SLAM) is a computer vision problem that tries to perform both tasks at the same time. For the case of an endoscopic procedure, the input to the SLAM algorithm is a monocular video sequence that is captured during exploration. Usually, Visual SLAM (VSLAM) algorithms are based on image matches i.e. they try to find parts of the environment that appear on two or more frames. In this Master Thesis, we explore and build on optical flow estimation, that is, algorithms that attempt to compute how each pixel moves between two images. Pixel motion is able to give dense correspondences for a video sequence. Most existing methods assume that the brightness of each pixel is constant regardless of where the camera is located. This is a good choice in most cases, for example, in outdoor scenes where diffuse objects are illuminated by the sun. In an endoscopy, the light and camera are co-located: changes in the position of the camera are correlated with changes of illumination. Moving the camera closer to an object makes it appear brighter. Our work explores two approaches to solve this problem. First, we develop a photometric model of light transport with a co-located light and camera. We introduce this model in an existing estimation algorithm and we are able to extract more precise image matches along with additional information from depth and surface normals. Secondly, we explore learning-based approaches. They have the great advantage of not requiring a hand-crafted illumination model, and their high-dimensional parameters are able to be learned using a large amount of training data. With the current technology, it is not possible to obtain enough ground truth optical flow in real endoscopy sequences. We explore different simulation environments and find that using a combination of real and synthetic data is key. With this, we obtain a 40% error reduction on optical flow estimation when evaluating on simulated data, and a 15% on captured data. Additionally, we show that mixing both types of training data produces much better qualitative results for other scene points whose ground truth is not available.

Tipo de Trabajo Académico: Trabajo Fin de Master