Omnidirectional multi-view systems.pdf

Omnidirectional multi-view systems: Calibration, features and 3D information

Puig Morales, Luis
Guerrero Campo, José Jesús (dir.)

Universidad de Zaragoza, 2011

Resumen: In this thesis we focus on omnidirectional multi-view systems, in particular central catadioptric systems. We go from the very early step of calibration to the high level task of 3D information recovery. We also consider a very important intermediate step, which is the development of properly adapted features for central catadioptric systems. In the following paragraphs we describe in moredetail the goals reached on this thesis. We selected the sphere camera model to deal with the central catadioptric systems since it gives information about the elements of the system than other models. In particular it gives important information about the mirror shape used in the catadioptric system. We introduce this model in detail in Chapter 2 along with an analysis of the relation between this model and actual central catadioptric systems. We also introduce the so-called lifted coordinates. In Sturm and Barreto (2008) the complete general theoretic projection matrix that considers all central catadioptric systems has been presented. We use this projection matrix to construct a new approach to calibrate any single- viewpoint catadioptric camera. Chapter 3 presents this approach, which requires twenty 3D-2D correspondences to compute the projection matrix. The 3D points must lie on at least three different planes. The projection of 3D points on a catadioptric image is performed linearly using a 6 × 10 projection matrix, which uses lifted coordinates for image and 3D points. From this matrix, an initial estimation of the intrinsic and extrinsic parameters of the catadioptric system is obtained. We use these parameters to initialize a non-linear optimization process. This approach is also able to calibrate slightly non-central cameras, in particular, fish-eye cameras. Since reprojection error is not sufficient to determine the accuracy of the approach, we decide to perform a 3D reconstruction from two omnidirectional images. During the development of our proposed calibration algorithm we realize that there was a lack of deep analysis and comparison of the existing calibration methods for central catadioptric systems. Moreover, for the robotics community where most tasks require to recover information from the environment, calibration of cameras is a basic step. At the same time this step should be easy to perform and reliable. In this order we perform a classification of the existing approaches. On the other hand we select those approaches which are already available as OpenSource and which do not require a complex pattern or scene to perform a comparison using synthetic and real images, so the user could select the more convenient for its particular case. Chapter 4 presents these methods and an analysis of their advantages and drawbacks. In Chapter 5 we present a deep analysis of the two-view relations of uncalibrated central catadioptric systems. We particularly pay attention to the mixture of central catadioptric systems and perspective cameras, which we call hybrid. The two-view geometric relations we consider are the hybrid fundamental matrix and the hybrid planar homography. These matrices contain useful geometric information. We study three different types of matrices, varying in complexity depending on their capacity to deal with a single or multiple types of central catadioptric systems. The first and simplest one is designed to deal with paracatadioptric systems, the second one and more complex, considers the combination of a perspective camera and any central catadioptric system. The last one is the complete and generic model which is able to deal with any combination of central catadioptric systems. We show that the generic and most complex model sometimes is not the best option when we deal with real images. Simpler models are not as accurate as the complete model in the ideal case, but they provide a better and more accurate behavior in presence of noise, being simpler and requiring less correspondences to be computed. Finally using the best models we present the successful matching between perspective images and hypercatadioptric images introducing geometrical constraints into a robust estimation technique. Another basic step in vision and robotics applications is feature detection/extraction. Through the years several techniques have been developed for conventional (perspective) cameras. The SIFT proposed by Lowe (2004) has become the most used feature extraction approach. This scale invariant approach is based on the approximation to the Laplacian of Gaussians (LoG) through the difference of Gaussians (DoG). The Gaussian filtering in Euclidean computer vision, which is required to construct the scale space of the images can be computed in two ways: either using convolution with a Gaussian kernel or by implementing the linear heat flow. In Chapter 6 we develop a new approach to compute the scale space of any omnidirectional image acquired with a central catadioptric system. We combine the sphere camera model and the partial differential equations framework on manifolds, to compute the Laplace-Beltrami (LB) operator which is a second order differential operator required to perform the Gaussian smoothing on catadioptric images. In Chapters 7, 8, 9 we present three applications we have developed using multi-view omnidirectional systems. The first one performs the self-localization of a robot using reference omnidirectional images in an indoors environment. We only need one omnidirectional image of the whole scene stored in the robot memory and a conventional uncalibrated on-board camera. We match the omnidirectional image and the conventional images captured by the on-board camera and compute the hybrid epipolar geometry using lifted coordinates and robust techniques. We map the epipole in the reference omnidirectional image to a ground plane through a homography in lifted coordinates also, giving the position of the robot in the planar ground, and its uncertainty. The second application consist of computing the orientation of a hand-held catadioptric system. We use the vanishing points which contain geometric information related to the orientation of the catadioptric system with respect to the dominant directions in man-made environments. The vanishing points are computed from the intersection of parallel lines. The 3D lines are projected in catadioptric images as conics, actually degenerate conics. We extract analytically the projected lines corresponding to straight lines in the scene by using the internal calibration and two image points that lie on the corresponding line projection. The last application instead of using a catadioptric system it uses a LadyBug1 camera, which corresponds to the Polycamera previously described. The unified coordinate frame as well as the intrinsic parameters of each camera are known. We develop a new way to compute a topological map using only orientation information. Similar to the last application but with a calibrated camera we exploit the natural presence of lines in man-made environments in dominant directions. From the parallel lines we robustly compute the three dominant directions using vanishing points. With this information we align the camera with respect to the scene and we identify the turns in the trajectory, assuming a Manhattan world where the changes of heading in are related by multiples of 90 degrees. We also use geometrical image-pair constraints as a tool to identify the visual traversable nodes that compose the topological map.