Music-driven Dance Regeneration with Controllable Key Pose Constraints
In this paper, we propose a novel framework for music-driven dance motion synthesis with controllable key pose constraint. In contrast to methods that generate dance motion sequences only based on music without any other controllable conditions, this work targets on synthesizing high-quality dance m...
Saved in:
Main Authors | , |
---|---|
Format | Journal Article |
Language | English |
Published |
08.07.2022
|
Subjects | |
Online Access | Get full text |
Cover
Loading…
Summary: | In this paper, we propose a novel framework for music-driven dance motion
synthesis with controllable key pose constraint. In contrast to methods that
generate dance motion sequences only based on music without any other
controllable conditions, this work targets on synthesizing high-quality dance
motion driven by music as well as customized poses performed by users. Our
model involves two single-modal transformer encoders for music and motion
representations and a cross-modal transformer decoder for dance motions
generation. The cross-modal transformer decoder achieves the capability of
synthesizing smooth dance motion sequences, which keeps a consistency with key
poses at corresponding positions, by introducing the local neighbor position
embedding. Such mechanism makes the decoder more sensitive to key poses and the
corresponding positions. Our dance synthesis model achieves satisfactory
performance both on quantitative and qualitative evaluations with extensive
experiments, which demonstrates the effectiveness of our proposed method. |
---|---|
DOI: | 10.48550/arxiv.2207.03682 |