All-photon Polarimetric Time-of-Flight Imaging

Seung Hwan Baek, Felix Heide

Research output: Chapter in Book/Report/Conference proceedingConference contribution

1 Scopus citations


Time-of-flight (ToF) sensors provide an image modal-ity fueling diverse applications, including LiDAR in au-tonomous driving, robotics, and augmented reality. Con-ventional ToF imaging methods estimate depth by sending pulses of light into a scene and measuring the ToF of the first-arriving photons directly reflected from a scene surface without any temporal delay. As such, all photons following this first response are typically considered as unwanted noise. In this paper, we depart from the principle of using first-arriving photons and propose an all-photon ToF imaging method that relies on the temporal-polarimetric analysis of first- and late-arriving photons which encode rich scene information in terms of geometry and material. To this end, we propose a novel temporal-polarimetric re-flectance model, an efficient capture method, and a reconstruction method that exploits the temporal-polarimetric changes of light reflected by the surface and sub-surface reflection. The proposed all-photon polarimetric ToF imaging method allows us to acquire depth, surface normals, and material parameters of a scene by utilizing all photons captured by the system, whereas conventional ToF imaging only obtains coarse depth from the first-arriving photons. We validate our method in simulation and experimentally with a prototype system.

Original languageEnglish (US)
Title of host publicationProceedings - 2022 IEEE/CVF Conference on Computer Vision and Pattern Recognition, CVPR 2022
PublisherIEEE Computer Society
Number of pages10
ISBN (Electronic)9781665469463
StatePublished - 2022
Event2022 IEEE/CVF Conference on Computer Vision and Pattern Recognition, CVPR 2022 - New Orleans, United States
Duration: Jun 19 2022Jun 24 2022

Publication series

NameProceedings of the IEEE Computer Society Conference on Computer Vision and Pattern Recognition
ISSN (Print)1063-6919


Conference2022 IEEE/CVF Conference on Computer Vision and Pattern Recognition, CVPR 2022
Country/TerritoryUnited States
CityNew Orleans

All Science Journal Classification (ASJC) codes

  • Software
  • Computer Vision and Pattern Recognition


  • Computational photography
  • Physics-based vision and shape-from-X


Dive into the research topics of 'All-photon Polarimetric Time-of-Flight Imaging'. Together they form a unique fingerprint.

Cite this