Learned hardware-in-the-loop phase retrieval for holographic near-eye displays

Praneeth Chakravarthula, Ethan Tseng, Tarun Srivastava, Henry Fuchs, Felix Heide

Research output: Contribution to journalArticlepeer-review

81 Scopus citations


Holography is arguably the most promising technology to provide wide field-of-view compact eyeglasses-style near-eye displays for augmented and virtual reality. However, the image quality of existing holographic displays is far from that of current generation conventional displays, effectively making today's holographic display systems impractical. This gap stems predominantly from the severe deviations in the idealized approximations of the "unknown"light transport model in a real holographic display, used for computing holograms. In this work, we depart from such approximate "ideal"coherent light transport models for computing holograms. Instead, we learn the deviations of the real display from the ideal light transport from the images measured using a display-camera hardware system. After this unknown light propagation is learned, we use it to compensate for severe aberrations in real holographic imagery. The proposed hardware-in-the-loop approach is robust to spatial, temporal and hardware deviations, and improves the image quality of existing methods qualitatively and quantitatively in SNR and perceptual quality. We validate our approach on a holographic display prototype and show that the method can fully compensate unknown aberrations and erroneous and non-linear SLM phase delays, without explicitly modeling them. As a result, the proposed method significantly outperforms existing state-of-the-art methods in simulation and experimentation - just by observing captured holographic images.

Original languageEnglish (US)
Article number186
JournalACM Transactions on Graphics
Issue number6
StatePublished - Nov 26 2020

All Science Journal Classification (ASJC) codes

  • Computer Graphics and Computer-Aided Design


  • artificial intelligence
  • augmented reality
  • computational displays
  • computer generated holography
  • deep learning
  • machine learning
  • near-eye displays
  • virtual reality


Dive into the research topics of 'Learned hardware-in-the-loop phase retrieval for holographic near-eye displays'. Together they form a unique fingerprint.

Cite this