Shape estimation from shading, defocus, and correspondence using light-field angular coherence

Michael W. Tao, Pratul P. Srinivasan, Sunil Hadap, Szymon Rusinkiewicz, Jitendra Malik, Ravi Ramamoorthi

Research output: Contribution to journalReview articlepeer-review

37 Scopus citations

Abstract

Light-field cameras are quickly becoming commodity items, with consumer and industrial applications. They capture many nearby views simultaneously using a single image with a micro-lens array, thereby providing a wealth of cues for depth recovery: Defocus, correspondence, and shading. In particular, apart from conventional image shading, one can refocus images after acquisition, and shift one's viewpoint within the sub-apertures of the main lens, effectively obtaining multiple views. We present a principled algorithm for dense depth estimation that combines defocus and correspondence metrics. We then extend our analysis to the additional cue of shading, using it to refine fine details in the shape. By exploiting an all-in-focus image, in which pixels are expected to exhibit angular coherence, we define an optimization framework that integrates photo consistency, depth consistency, and shading consistency. We show that combining all three sources of information: Defocus, correspondence, and shading, outperforms state-of-the-art light-field depth estimation algorithms in multiple scenarios.

Original languageEnglish (US)
Article number7452621
Pages (from-to)546-560
Number of pages15
JournalIEEE Transactions on Pattern Analysis and Machine Intelligence
Volume39
Issue number3
DOIs
StatePublished - Mar 2017

All Science Journal Classification (ASJC) codes

  • Software
  • Computer Vision and Pattern Recognition
  • Computational Theory and Mathematics
  • Artificial Intelligence
  • Applied Mathematics

Keywords

  • 3D reconstruction
  • Light fields
  • depth cues
  • reflection components separation
  • shape from shading
  • specular-free image

Fingerprint Dive into the research topics of 'Shape estimation from shading, defocus, and correspondence using light-field angular coherence'. Together they form a unique fingerprint.

Cite this