All-photon Polarimetric Time-of-Flight Imaging

Time-of-flight (ToF) sensors provide an image modal-ity fueling diverse applications, including LiDAR in au-tonomous driving, robotics, and augmented reality. Con-ventional ToF imaging methods estimate depth by sending pulses of light into a scene and measuring the ToF of the first-arriving photons directly reflected from a scene surface without any temporal delay. As such, all photons following this first response are typically considered as unwanted noise. In this paper, we depart from the principle of using first-arriving photons and propose an all-photon ToF imaging method that relies on the temporal-polarimetric analysis of first- and late-arriving photons which encode rich scene information in terms of geometry and material. To this end, we propose a novel temporal-polarimetric re-flectance model, an efficient capture method, and a reconstruction method that exploits the temporal-polarimetric changes of light reflected by the surface and sub-surface reflection. The proposed all-photon polarimetric ToF imaging method allows us to acquire depth, surface normals, and material parameters of a scene by utilizing all photons captured by the system, whereas conventional ToF imaging only obtains coarse depth from the first-arriving photons. We validate our method in simulation and experimentally with a prototype system.