Keyframe Control of Music-Driven 3D Dance Generation
文献类型：期刊论文
| 作者 | Yang, Zhipeng1,2; Wen, Yu-Hui3; Chen, Shu-Yu1,2; Liu, Xiao4; Gao, Yuan4; Liu, Yong-Jin3; Gao, Lin1,2; Fu, Hongbo5 |
| 刊名 | IEEE TRANSACTIONS ON VISUALIZATION AND COMPUTER GRAPHICS
 |
| 出版日期 | 2024-07-01 |
| 卷号 | 30期号:7页码:3474-3486 |
| 关键词 | Humanities Animation Three-dimensional displays Deep learning Probabilistic logic Interpolation Task analysis 3D animation choreography generative flows multi-modal music-driven |
| ISSN号 | 1077-2626 |
| DOI | 10.1109/TVCG.2023.3235538 |
| 英文摘要 | For 3D animators, choreography with artificial intelligence has attracted more attention recently. However, most existing deep learning methods mainly rely on music for dance generation and lack sufficient control over generated dance motions. To address this issue, we introduce the idea of keyframe interpolation for music-driven dance generation and present a novel transition generation technique for choreography. Specifically, this technique synthesizes visually diverse and plausible dance motions by using normalizing flows to learn the probability distribution of dance motions conditioned on a piece of music and a sparse set of key poses. Thus, the generated dance motions respect both the input musical beats and the key poses. To achieve a robust transition of varying lengths between the key poses, we introduce a time embedding at each timestep as an additional condition. Extensive experiments show that our model generates more realistic, diverse, and beat-matching dance motions than the compared state-of-the-art methods, both qualitatively and quantitatively. Our experimental results demonstrate the superiority of the keyframe-based control for improving the diversity of the generated dance motions. |
| 资助项目 | National Key R&D Program of China[2020AAA0104500] ; National Natural Science Foundation of China[62102403] ; National Natural Science Foundation of China[61725204] ; National Natural Science Foundation of China[62202257] ; Beijing Natural Science Foundation[L222008] ; Beijing Municipal Natural Science Foundation for Distinguished Young Scholars[JQ21013] ; China Postdoctoral Science Foundation[2021M701891] ; China Postdoctoral Science Foundation[2022M713205] ; Youth Innovation Promotion Association CAS |
| WOS研究方向 | Computer Science |
| 语种 | 英语 |
| WOS记录号 | WOS:001258936700020 |
| 出版者 | IEEE COMPUTER SOC |
| 源URL | [http://119.78.100.204/handle/2XEOYT63/39830]  |
| 专题 | 中国科学院计算技术研究所期刊论文_英文 |
| 通讯作者 | Liu, Yong-Jin; Gao, Lin |
| 作者单位 | 1.Chinese Acad Sci, Beijing Key Lab Mobile Comp & Pervas Device, Inst Comp Technol, Beijing 100190, Peoples R China 2.Univ Chinese Acad Sci, Beijing 100190, Peoples R China 3.Tsinghua Univ, CS Dept, BNRist, Beijing 100190, Peoples R China 4.Tomorrow Adv Life Educ Grp, Beijing 100190, Peoples R China 5.City Univ Hong Kong, Sch Creat Media, Hong Kong, Peoples R China |
| 推荐引用方式 GB/T 7714 | Yang, Zhipeng,Wen, Yu-Hui,Chen, Shu-Yu,et al. Keyframe Control of Music-Driven 3D Dance Generation[J]. IEEE TRANSACTIONS ON VISUALIZATION AND COMPUTER GRAPHICS,2024,30(7):3474-3486. |
| APA | Yang, Zhipeng.,Wen, Yu-Hui.,Chen, Shu-Yu.,Liu, Xiao.,Gao, Yuan.,...&Fu, Hongbo.(2024).Keyframe Control of Music-Driven 3D Dance Generation.IEEE TRANSACTIONS ON VISUALIZATION AND COMPUTER GRAPHICS,30(7),3474-3486. |
| MLA | Yang, Zhipeng,et al."Keyframe Control of Music-Driven 3D Dance Generation".IEEE TRANSACTIONS ON VISUALIZATION AND COMPUTER GRAPHICS 30.7(2024):3474-3486. |
入库方式： OAI收割
来源：计算技术研究所
浏览0
下载0
收藏0
其他版本
除非特别说明，本系统中所有内容都受版权保护，并保留所有权利。