HINT: Learning Complete Human Neural Representations from Limited Viewpoints

Alessandro Sanvito, Andrea Ramazzina, Stefanie Walz, Mario Bijelic, Felix Heide

Research output: Chapter in Book/Report/Conference proceedingConference contribution

Abstract

No augmented application is possible without animated humanoid avatars. At the same time, generating human replicas from real-world monocular hand-held or robotic sensor setups is challenging due to the limited availability of views. Previous work showed the feasibility of virtual avatars but required the presence of 360° views of the targeted subject. To address this issue, we propose HINT, a NeRF-based algorithm able to learn a detailed and complete human model from limited viewing angles. We achieve this by introducing a symmetry prior, regularization constraints, and training cues from large human datasets. In particular, we introduce a sagittal plane symmetry prior to the appearance of the human, directly supervise the density function of the human model using explicit 3D body modeling, and leverage a co-learned human digitization network as additional supervision for the unseen angles.As a result, our method can reconstruct complete humans even from a few viewing angles, increasing performance by more than 15% PSNR compared to previous state-of-the-art algorithms.

Original languageEnglish (US)
Title of host publication35th IEEE Intelligent Vehicles Symposium, IV 2024
PublisherInstitute of Electrical and Electronics Engineers Inc.
Pages1556-1563
Number of pages8
ISBN (Electronic)9798350348811
DOIs
StatePublished - 2024
Event35th IEEE Intelligent Vehicles Symposium, IV 2024 - Jeju Island, Korea, Republic of
Duration: Jun 2 2024Jun 5 2024

Publication series

NameIEEE Intelligent Vehicles Symposium, Proceedings
ISSN (Print)1931-0587
ISSN (Electronic)2642-7214

Conference

Conference35th IEEE Intelligent Vehicles Symposium, IV 2024
Country/TerritoryKorea, Republic of
CityJeju Island
Period6/2/246/5/24

All Science Journal Classification (ASJC) codes

  • Computer Science Applications
  • Automotive Engineering
  • Modeling and Simulation

Fingerprint

Dive into the research topics of 'HINT: Learning Complete Human Neural Representations from Limited Viewpoints'. Together they form a unique fingerprint.

Cite this