Pose2Pose

Nora S. Willett, Hijung Valentina Shin, Zeyu Jin, Wilmot Li, Adam Finkelstein

Research output: Chapter in Book/Report/Conference proceedingConference contribution

Abstract

An artist faces two challenges when creating a 2D animated character to mimic a specific human performance. First, the artist must design and draw a collection of artwork depicting portions of the character in a suitable set of poses, for example arm and hand poses that can be selected and combined to express the range of gestures typical for that person. Next, to depict a specific performance, the artist must select and position the appropriate set of artwork at each moment of the animation. This paper presents a system that addresses these challenges by leveraging video of the target human performer. Our system tracks arm and hand poses in an example video of the target. The UI displays clusters of these poses to help artists select representative poses that capture the actor's style and personality. From this mapping of pose data to character artwork, our system can generate an animation from a new performance video. It relies on a dynamic programming algorithm to optimize for smooth animations that match the poses found in the video. Artists used our system to create four 2D characters and were pleased with the final automatically animated results. We also describe additional applications addressing audio-driven or text-based animations.

Original languageEnglish (US)
Title of host publicationProceedings of the 25th International Conference on Intelligent User Interfaces, IUI 2020
PublisherAssociation for Computing Machinery
Pages88-99
Number of pages12
ISBN (Electronic)9781450371186
DOIs
StatePublished - Mar 17 2020
Event25th ACM International Conference on Intelligent User Interfaces, IUI 2020 - Cagliari, Italy
Duration: Mar 17 2020Mar 20 2020

Publication series

NameInternational Conference on Intelligent User Interfaces, Proceedings IUI

Conference

Conference25th ACM International Conference on Intelligent User Interfaces, IUI 2020
CountryItaly
CityCagliari
Period3/17/203/20/20

All Science Journal Classification (ASJC) codes

  • Software
  • Human-Computer Interaction

Keywords

  • 2D character creation
  • animation
  • pose selection

Fingerprint Dive into the research topics of 'Pose2Pose'. Together they form a unique fingerprint.

  • Cite this

    Willett, N. S., Shin, H. V., Jin, Z., Li, W., & Finkelstein, A. (2020). Pose2Pose. In Proceedings of the 25th International Conference on Intelligent User Interfaces, IUI 2020 (pp. 88-99). (International Conference on Intelligent User Interfaces, Proceedings IUI). Association for Computing Machinery. https://doi.org/10.1145/3377325.3377505