中国科学院机构知识库网格
Chinese Academy of Sciences Institutional Repositories Grid
首页 机构 成果 学者
  
  1. CAS IR Grid
  2. 计算技术研究所
  3. 中国科学院计算技术研究所
  4. 中国科学院计算技术研究所期刊论文
  5. 英文
High-Quality Video Generation from Static Structural Annotations

文献类型：期刊论文

作者Sheng, Lu1; Pan, Junting2; Guo, Jiaming3; Shao, Jing4; Loy, Chen Change5
刊名INTERNATIONAL JOURNAL OF COMPUTER VISION
出版日期2020-05-28
页码18
ISSN号0920-5691
关键词Unsupervised learning Conditioned generative model Image and video synthesis Motion prediction and estimatiovn
DOI10.1007/s11263-020-01334-x
英文摘要This paper proposes a novel unsupervised video generation that is conditioned on a single structural annotation map, which in contrast to prior conditioned video generation approaches, provides a good balance between motion flexibility and visual quality in the generation process. Different from end-to-end approaches that model the scene appearance and dynamics in a single shot, we try to decompose this difficult task into two easier sub-tasks in a divide-and-conquer fashion, thus achieving remarkable results overall. The first sub-task is an image-to-image (I2I) translation task that synthesizes high-quality starting frame from the input structural annotation map. The second image-to-video (I2V) generation task applies the synthesized starting frame and the associated structural annotation map to animate the scene dynamics for the generation of a photorealistic and temporally coherent video. We employ a cycle-consistent flow-based conditioned variational autoencoder to capture the long-term motion distributions, by which the learned bi-directional flows ensure the physical reliability of the predicted motions and provide explicit occlusion handling in a principled manner. Integrating structural annotations into the flow prediction also improves the structural awareness in the I2V generation process. Quantitative and qualitative evaluations over the autonomous driving and human action datasets demonstrate the effectiveness of the proposed approach over the state-of-the-art methods. The code has been released:.
资助项目National Natural Science Foundation of China[61906012] ; Singapore MOE AcRF Tier 1[2018-T1-002-056] ; NTU NAP ; NTU SUG
WOS研究方向Computer Science
语种英语
出版者SPRINGER
WOS记录号WOS:000554776700001
源URL[http://119.78.100.204/handle/2XEOYT63/15890]  
专题中国科学院计算技术研究所期刊论文_英文
通讯作者Sheng, Lu
作者单位1.Beihang Univ, Coll Software, Beijing, Peoples R China
2.Chinese Univ Hong Kong, CUHK SenseTime Joint Lab, Hong Kong, Peoples R China
3.Chinese Acad Sci, Inst Comp Technol, Beijing, Peoples R China
4.SenseTime Res, Shenzhen, Guangdong, Peoples R China
5.Nanyang Technol Univ, SenseTime NTU Joint Res Ctr, Singapore, Singapore
推荐引用方式
GB/T 7714		 Sheng, Lu,Pan, Junting,Guo, Jiaming,et al. High-Quality Video Generation from Static Structural Annotations[J]. INTERNATIONAL JOURNAL OF COMPUTER VISION,2020:18.
APA Sheng, Lu,Pan, Junting,Guo, Jiaming,Shao, Jing,&Loy, Chen Change.(2020).High-Quality Video Generation from Static Structural Annotations.INTERNATIONAL JOURNAL OF COMPUTER VISION,18.
MLA Sheng, Lu,et al."High-Quality Video Generation from Static Structural Annotations".INTERNATIONAL JOURNAL OF COMPUTER VISION (2020):18.

入库方式： OAI收割

来源：计算技术研究所

浏览0
下载0
收藏0
其他版本

除非特别说明，本系统中所有内容都受版权保护，并保留所有权利。