FlexISP: A flexible camera image processing framework

Felix Heide, Markus Steinberger, Yun Ta Tsai, Mushfiqur Rouf, Dawid Paja¸k, Dikpal Reddy, Orazio Gallo, Jing Liu, Wolfgang Heidrich, Karen Egiazarian, Jan Kautz, Kari Pulli

Research output: Contribution to journalArticle

110 Scopus citations

Abstract

Conventional pipelines for capturing, displaying, and storing images are usually defined as a series of cascaded modules, each responsible for addressing a particular problem. While this divide-and-conquer approach offers many benefits, it also introduces a cumulative error, as each step in the pipeline only considers the output of the previous step, not the original sensor data. We propose an end-to-end system that is aware of the camera and image model, enforces naturalimage priors, while jointly accounting for common image processing steps like demosaicking, denoising, deconvolution, and so forth, all directly in a given output representation (e.g., YUV, DCT). Our system is flexible and we demonstrate it on regular Bayer images as well as images from custom sensors. In all cases, we achieve large improvements in image quality and signal reconstruction compared to state-of-the-art techniques. Finally, we show that our approach is capable of very efficiently handling high-resolution images, making even mobile implementations feasible.(Figure Presented).

Original languageEnglish (US)
JournalACM Transactions on Graphics
Volume33
Issue number6
DOIs
StatePublished - Nov 19 2014
Externally publishedYes

All Science Journal Classification (ASJC) codes

  • Computer Graphics and Computer-Aided Design

Keywords

  • Image processing
  • Image reconstruction

Fingerprint Dive into the research topics of 'FlexISP: A flexible camera image processing framework'. Together they form a unique fingerprint.

  • Cite this

    Heide, F., Steinberger, M., Tsai, Y. T., Rouf, M., Paja¸k, D., Reddy, D., Gallo, O., Liu, J., Heidrich, W., Egiazarian, K., Kautz, J., & Pulli, K. (2014). FlexISP: A flexible camera image processing framework. ACM Transactions on Graphics, 33(6). https://doi.org/10.1145/2661229.2661260