MOCHA: Real-Time Motion Characterization via Context Matching

• Published in: arXiv.org
• Main Authors: Jang, Deok-Kyeong, Ye, Yuting, Jungdam Won, Lee, Sung-Hee
• Format: Paper Journal Article
• Language: English
• Published: Ithaca Cornell University Library, arXiv.org 16.10.2023
• Subjects: Ablation; Animation; Autoregressive models; Body parts; Computer Science - Artificial Intelligence; Computer Science - Graphics; Context; Real time; Topology
• ISSN: 2331-8422
• DOI: 10.48550/arxiv.2310.10079

Transforming neutral, characterless input motions to embody the distinct style of a notable character in real time is highly compelling for character animation. This paper introduces MOCHA, a novel online motion characterization framework that transfers both motion styles and body proportions from a target character to an input source motion. MOCHA begins by encoding the input motion into a motion feature that structures the body part topology and captures motion dependencies for effective characterization. Central to our framework is the Neural Context Matcher, which generates a motion feature for the target character with the most similar context to the input motion feature. The conditioned autoregressive model of the Neural Context Matcher can produce temporally coherent character features in each time frame. To generate the final characterized pose, our Characterizer network incorporates the characteristic aspects of the target motion feature into the input motion feature while preserving its context. This is achieved through a transformer model that introduces the adaptive instance normalization and context mapping-based cross-attention, effectively injecting the character feature into the source feature. We validate the performance of our framework through comparisons with prior work and an ablation study. Our framework can easily accommodate various applications, including characterization with only sparse input and real-time characterization. Additionally, we contribute a high-quality motion dataset comprising six different characters performing a range of motions, which can serve as a valuable resource for future research.