Lightfield completion

A light field is a 4D function representing radiance as a function of ray position and direction in 3D space. In this paper we describe a method for recovering gaps in light fields of scenes that contain significant occluders. In these situations, although a large fraction of the scene may be blocked in any one view, most scene points are visible in at least some views. As a consequence, although too much information is missing to employ 2D completion methods that operate within a single view, it may be possible to recover the lost information by completion in 4D-the full dimensionality of the light field. The proposed light field completion method has three main steps: Registration, initial estimation, and high dimensional texture synthesis and/or inpainting. At the registration stage, the set of images are shifted and re-projected so that the corresponding pixels from different images are aligned in the reconstructed light field. Following this, the estimation step uses a naive technique to fill-in parts of gaps using the available information from the multiple images. This serves as the initial condition for the next and last step, where the missing information is recovered via high dimensional texture synthesis and/or inpainting. These two steps of initial condition and completion are iterated. The algorithm is illustrated with real examples.