Interactive 3D Modeling with a Generative Adversarial Network

Jerry Liu, Fisher Yu, Thomas Funkhouser

Research output: Chapter in Book/Report/Conference proceedingConference contribution

20 Scopus citations

Abstract

We propose the idea of using a generative adversarial network (GAN) to assist users in designing real-world shapes with a simple interface. Users edit a voxel grid with a Minecraft-like interface. Yet they can execute a SNAP command at any time, which transforms their rough model into a desired shape that is both similar and realistic. They can edit and snap until they are satisfied with the result. The advantage of this approach is to assist novice users to create 3D models characteristic of the training data by only specifying rough edits. Our key contribution is to create a suitable projection operator around a 3D-GAN that maps an arbitrary 3D voxel input to a latent vector in the shape manifold of the generator that is both similar in shape to the input but also realistic. Experiments show our method is promising for computer-Assisted interactive modeling.

Original languageEnglish (US)
Title of host publicationProceedings - 2017 International Conference on 3D Vision, 3DV 2017
PublisherInstitute of Electrical and Electronics Engineers Inc.
Pages126-134
Number of pages9
ISBN (Electronic)9781538626108
DOIs
StatePublished - May 25 2018
Event7th IEEE International Conference on 3D Vision, 3DV 2017 - Qingdao, China
Duration: Oct 10 2017Oct 12 2017

Publication series

NameProceedings - 2017 International Conference on 3D Vision, 3DV 2017

Other

Other7th IEEE International Conference on 3D Vision, 3DV 2017
CountryChina
CityQingdao
Period10/10/1710/12/17

All Science Journal Classification (ASJC) codes

  • Media Technology
  • Computer Vision and Pattern Recognition
  • Signal Processing

Keywords

  • Computer-Graphics
  • GAN
  • Interactive-Modeling
  • Voxel

Fingerprint Dive into the research topics of 'Interactive 3D Modeling with a Generative Adversarial Network'. Together they form a unique fingerprint.

  • Cite this

    Liu, J., Yu, F., & Funkhouser, T. (2018). Interactive 3D Modeling with a Generative Adversarial Network. In Proceedings - 2017 International Conference on 3D Vision, 3DV 2017 (pp. 126-134). (Proceedings - 2017 International Conference on 3D Vision, 3DV 2017). Institute of Electrical and Electronics Engineers Inc.. https://doi.org/10.1109/3DV.2017.00024