TY - JOUR
T1 - Single-shot monocular RGB-D imaging using uneven double refraction
AU - Meuleman, Andreas
AU - Baek, Seung Hwan
AU - Heide, Felix
AU - Kim, Min H.
N1 - Funding Information:
Min H. Kim acknowledges Korea NRF grants (2019R1A2C3007229, 2013M3A6A6073718), KOCCA in MCST of Korea, Samsung Research, and Cross-Ministry Giga KOREA (GK17P0200).
Publisher Copyright:
© 2020 IEEE.
PY - 2020
Y1 - 2020
N2 - Cameras that capture color and depth information have become an essential imaging modality for applications in robotics, autonomous driving, virtual, and augmented reality. Existing RGB-D cameras rely on multiple sensors or active illumination with specialized sensors. In this work, we propose a method for monocular single-shot RGB-D imaging. Instead of learning depth from single-image depth cues, we revisit double-refraction imaging using a birefractive medium, measuring depth as the displacement of differently refracted images superimposed in a single capture. However, existing double-refraction methods are orders of magnitudes too slow to be used in real-time applications, e.g., in robotics, and provide only inaccurate depth due to correspondence ambiguity in double reflection. We resolve this ambiguity optically by leveraging the orthogonality of the two linearly polarized rays in double refraction - introducing uneven double refraction by adding a linear polarizer to the birefractive medium. Doing so makes it possible to develop a real-time method for reconstructing sparse depth and color simultaneously in real-time. We validate the proposed method, both synthetically and experimentally, and demonstrate 3D object detection and photographic applications.
AB - Cameras that capture color and depth information have become an essential imaging modality for applications in robotics, autonomous driving, virtual, and augmented reality. Existing RGB-D cameras rely on multiple sensors or active illumination with specialized sensors. In this work, we propose a method for monocular single-shot RGB-D imaging. Instead of learning depth from single-image depth cues, we revisit double-refraction imaging using a birefractive medium, measuring depth as the displacement of differently refracted images superimposed in a single capture. However, existing double-refraction methods are orders of magnitudes too slow to be used in real-time applications, e.g., in robotics, and provide only inaccurate depth due to correspondence ambiguity in double reflection. We resolve this ambiguity optically by leveraging the orthogonality of the two linearly polarized rays in double refraction - introducing uneven double refraction by adding a linear polarizer to the birefractive medium. Doing so makes it possible to develop a real-time method for reconstructing sparse depth and color simultaneously in real-time. We validate the proposed method, both synthetically and experimentally, and demonstrate 3D object detection and photographic applications.
UR - http://www.scopus.com/inward/record.url?scp=85094174877&partnerID=8YFLogxK
UR - http://www.scopus.com/inward/citedby.url?scp=85094174877&partnerID=8YFLogxK
U2 - 10.1109/CVPR42600.2020.00254
DO - 10.1109/CVPR42600.2020.00254
M3 - Conference article
AN - SCOPUS:85094174877
SN - 1063-6919
SP - 2462
EP - 2471
JO - Proceedings of the IEEE Computer Society Conference on Computer Vision and Pattern Recognition
JF - Proceedings of the IEEE Computer Society Conference on Computer Vision and Pattern Recognition
M1 - 9157031
T2 - 2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition, CVPR 2020
Y2 - 14 June 2020 through 19 June 2020
ER -