Differentiable Point-Based Radiance Fields for Efficient View Synthesis

Qiang Zhang, Seung Hwan Baek, Szymon Rusinkiewicz, Felix Heide

Research output: Chapter in Book/Report/Conference proceedingConference contribution

8 Scopus citations

Abstract

We propose a differentiable rendering algorithm for efficient novel view synthesis. By departing from volume-based representations in favor of a learned point representation, we improve on existing methods more than an order of magnitude in memory and runtime, both in training and inference. The method begins with a uniformly-sampled random point cloud and learns per-point position and view-dependent appearance, using a differentiable splat-based renderer to train the model to reproduce a set of input training images with the given pose. Our method is up to 300 × faster than NeRF in both training and inference, with only a marginal sacrifice in quality, while using less than 10 MB of memory for a static scene. For dynamic scenes, our method trains two orders of magnitude faster than STNeRF and renders at a near interactive rate, while maintaining high image quality and temporal coherence even without imposing any temporal-coherency regularizers.

Original languageEnglish (US)
Title of host publicationProceedings - SIGGRAPH Asia 2022 Conference Papers
EditorsStephen N. Spencer
PublisherAssociation for Computing Machinery, Inc
ISBN (Electronic)9781450394703
DOIs
StatePublished - Nov 29 2022
EventSIGGRAPH Asia 2022 - Computer Graphics and Interactive Techniques Conference - Asia, SA 2022 - Daegu, Korea, Republic of
Duration: Dec 6 2022Dec 9 2022

Publication series

NameProceedings - SIGGRAPH Asia 2022 Conference Papers

Conference

ConferenceSIGGRAPH Asia 2022 - Computer Graphics and Interactive Techniques Conference - Asia, SA 2022
Country/TerritoryKorea, Republic of
CityDaegu
Period12/6/2212/9/22

All Science Journal Classification (ASJC) codes

  • Computer Graphics and Computer-Aided Design
  • Human-Computer Interaction

Keywords

  • Image-based Rendering
  • Neural Rendering
  • Novel View Synthesis

Fingerprint

Dive into the research topics of 'Differentiable Point-Based Radiance Fields for Efficient View Synthesis'. Together they form a unique fingerprint.

Cite this