Tutorial: "Underwater Image Processing and Photomosaicing" by UdG

Summary:

The first day started with the presentation by Dr. Ricard Campos of the contents of the seminar and an illustration of their applications in a practical context. This included the analysis of the requirements for carrying out the hands-on learning activities and ensuring that all the participants had installed in their computers all the libraries required to carry out the practical activities. Then, Dr. Rafael Garcia presented the challenges of Underwater Imaging. We saw how the image is formed underwater and what are the interactions between light and water. The participants understood that underwater images are affected by particular factors such as non-uniform illumination, caustics, blurring, suspended particles and scattering, making impossible the acquisition of images of large seafloor areas in a single shot. As a consequence, images can only be acquired from a short range, and commonly with the support of an artificial lighting source.

The programme on the first day continued with the analysis of feature detectors and descriptors by Rafael Garcia for automatically solving the correspondence problem between pairs of underwater images. The participants were exposed to the fact that given two or more images of the same 3D scene, taken from different points of view, the correspondence problem refers to the task of finding a set of points in one image which can be identified as the same points in another image. To do this, points or features in one image are detected, described using image descriptors and matched with the corresponding points or features in another image. To finish the first day, we addressed the problem of image registration: when an underwater robot takes images of the seafloor while carrying out a mission, the next step for mapping the explored area is to estimate the motion of the scene relative to the camera. In this part we analised the hierarchy of transformations with different degrees of freedom that can be derived from the estimation of homographies (or planar transformations). The lecture finished by describing the RANSAC algorithm for outlier rejection.

On the second day, Dr. Ricard Campos addressed the problem of Global Alignment. Building a large mosaic without human intervention is a difficult and challenging task that requires a robust (and fully automatic) estimation procedure. Reliable image registration between all possible pairs of overlapping images cannot be automatized. Moreover, tuning the weights for the optimization that generates a globally aligned mosaic is a difficult task, which requires some experience, and therefore a case by case approach is needed, with no possibility of providing a universally usable software, as in the case of navigation-based mosaics. In order to achieve this goal, the core routine is the estimation of the motion between two images, and this is done by robustly detecting the set of common (corresponding) points between pairs images, as detailed on the first day. Based on the results from the previous day, Dr. Campos noted that image processing on its own produces coherent local image alignment, but the trajectory estimated only from the images suffers from serious drift. Therefore, even when image processing is used to improve image alignment, overlapping non-consecutive image pairs should be taken into account to avoid drifts in the localization of the images in the long run.

We saw in this seminar that building a large globally-aligned photo-mosaic requires the detection of corresponding points in consecutive images, prediction of overlaps between non-consecutive images, and detection of corresponding points in non-consecutive images (known as loop detection). Then, a number of verifications should be run to reject any pair of images for which “ill-conditioned” correspondence points have been detected. Next, a large non-linear optimization function has to be solved to optimally combine image registration information and navigation data. This optimization recomputes the 3D position and orientation of every image, taking into account both the initial navigation data and the image processing constraints.

Finally, hands on exercises provided more insight into building a complete system for underwater photomosaicing following the tutorial documentation that was prepared for the participants.

Outcomes of the expert visit:

• Understand the complexities and peculiarities of underwater image formation.

• Explore alternatives for adequate image processing under the presence of different lighting challenges.   

• Learn and experiment with real images using a software pipeline that allowed to create globally-consistent underwater photomosaics.    

The tutorial allowed meeting many students, researchers and academic staff from both universities, as well as the exchange of ideas and research interests.