End-to-end optimization of optics and image processing for achromatic extended depth of field and super-resolution imaging

Vincent Sitzmann, Steven Diamond, Yifan Peng, Xiong Dun, Stephen Boyd, Wolfgang Heidrich, Felix Heide, Gordon Wetzstein

Research output: Contribution to journalArticle

25 Scopus citations

Abstract

In typical cameras the optical system is designed first; once it is fixed, the parameters in the image processing algorithm are tuned to get good image reproduction. In contrast to this sequential design approach, we consider joint optimization of an optical system (for example, the physical shape of the lens) together with the parameters of the reconstruction algorithm.We build a fully-differentiable simulation model that maps the true source image to the reconstructed one. The model includes diffractive light propagation, depth and wavelength-dependent effects, noise and nonlinearities, and the image post-processing. We jointly optimize the optical parameters and the image processing algorithm parameters so as to minimize the deviation between the true and reconstructed image, over a large set of images. We implement our joint optimization method using autodifferentiation to efficiently compute parameter gradients in a stochastic optimization algorithm. We demonstrate the efficacy of this approach by applying it to achromatic extended depth of field and snapshot super-resolution imaging.

Original languageEnglish (US)
Article number114
JournalACM Transactions on Graphics
Volume37
Issue number4
DOIs
StatePublished - Jan 1 2018
Externally publishedYes

All Science Journal Classification (ASJC) codes

  • Computer Graphics and Computer-Aided Design

Keywords

  • computational optics

Fingerprint Dive into the research topics of 'End-to-end optimization of optics and image processing for achromatic extended depth of field and super-resolution imaging'. Together they form a unique fingerprint.

  • Cite this