TY - GEN
T1 - HINT
T2 - 35th IEEE Intelligent Vehicles Symposium, IV 2024
AU - Sanvito, Alessandro
AU - Ramazzina, Andrea
AU - Walz, Stefanie
AU - Bijelic, Mario
AU - Heide, Felix
N1 - Publisher Copyright:
© 2024 IEEE.
PY - 2024
Y1 - 2024
N2 - No augmented application is possible without animated humanoid avatars. At the same time, generating human replicas from real-world monocular hand-held or robotic sensor setups is challenging due to the limited availability of views. Previous work showed the feasibility of virtual avatars but required the presence of 360° views of the targeted subject. To address this issue, we propose HINT, a NeRF-based algorithm able to learn a detailed and complete human model from limited viewing angles. We achieve this by introducing a symmetry prior, regularization constraints, and training cues from large human datasets. In particular, we introduce a sagittal plane symmetry prior to the appearance of the human, directly supervise the density function of the human model using explicit 3D body modeling, and leverage a co-learned human digitization network as additional supervision for the unseen angles.As a result, our method can reconstruct complete humans even from a few viewing angles, increasing performance by more than 15% PSNR compared to previous state-of-the-art algorithms.
AB - No augmented application is possible without animated humanoid avatars. At the same time, generating human replicas from real-world monocular hand-held or robotic sensor setups is challenging due to the limited availability of views. Previous work showed the feasibility of virtual avatars but required the presence of 360° views of the targeted subject. To address this issue, we propose HINT, a NeRF-based algorithm able to learn a detailed and complete human model from limited viewing angles. We achieve this by introducing a symmetry prior, regularization constraints, and training cues from large human datasets. In particular, we introduce a sagittal plane symmetry prior to the appearance of the human, directly supervise the density function of the human model using explicit 3D body modeling, and leverage a co-learned human digitization network as additional supervision for the unseen angles.As a result, our method can reconstruct complete humans even from a few viewing angles, increasing performance by more than 15% PSNR compared to previous state-of-the-art algorithms.
UR - http://www.scopus.com/inward/record.url?scp=85199804753&partnerID=8YFLogxK
UR - http://www.scopus.com/inward/citedby.url?scp=85199804753&partnerID=8YFLogxK
U2 - 10.1109/IV55156.2024.10588836
DO - 10.1109/IV55156.2024.10588836
M3 - Conference contribution
AN - SCOPUS:85199804753
T3 - IEEE Intelligent Vehicles Symposium, Proceedings
SP - 1556
EP - 1563
BT - 35th IEEE Intelligent Vehicles Symposium, IV 2024
PB - Institute of Electrical and Electronics Engineers Inc.
Y2 - 2 June 2024 through 5 June 2024
ER -