中国科学院机构知识库网格
Chinese Academy of Sciences Institutional Repositories Grid
首页 机构 成果 学者
  
  1. CAS IR Grid
  2. 计算技术研究所
  3. 中国科学院计算技术研究所
  4. 中国科学院计算技术研究所期刊论文
  5. 英文
DeepFaceVideoEditing: Sketch-based Deep Editing of Face Videos

文献类型：期刊论文

作者Liu, Feng-Lin2,3,4; Chen, Shu-Yu2,3,4; Lai, Yu-Kun1; Li, Chunpeng2,3,4; Jiang, Yue-Ren2,3,4; Fu, Hongbo5; Gao, Lin2,3,4
刊名ACM TRANSACTIONS ON GRAPHICS
出版日期2022-07-01
卷号41期号:4页码:16
关键词Video Editing Video Propagation Sketch-based Interaction
ISSN号0730-0301
DOI10.1145/3528223.3530056
英文摘要Sketches, which are simple and concise, have been used in recent deep image synthesis methods to allow intuitive generation and editing of facial images. However, it is nontrivial to extend such methods to video editing due to various challenges, ranging from appropriate manipulation propagation and fusion of multiple editing operations to ensure temporal coherence and visual quality. To address these issues, we propose a novel sketch-based facial video editing framework, in which we represent editing manipulations in latent space and propose specific propagation and fusion modules to generate high-quality video editing results based on StyleGAN3. Specifically, we first design an optimization approach to represent sketch editing manipulations by editing vectors, which are propagated to the whole video sequence using a proper strategy to cope with different editing needs. Specifically, input editing operations are classified into two categories: temporally consistent editing and temporally variant editing. The former (e.g., change of face shape) is applied to the whole video sequence directly, while the latter (e.g., change of facial expression or dynamics) is propagated with the guidance of expression or only affects adjacent frames in a given time window. Since users often perform different editing operations in multiple frames, we further present a region-aware fusion approach to fuse diverse editing effects. Our method supports video editing on facial structure and expression movement by sketch, which cannot be achieved by previous works. Both qualitative and quantitative evaluations show the superior editing ability of our system to existing and alternative solutions.
资助项目National Natural Science Foundation of China[62102403] ; National Natural Science Foundation of China[61872440] ; National Natural Science Foundation of China[62061136007] ; Beijing Municipal Natural Science Foundation for Distinguished Young Scholars[JQ21013] ; Youth Innovation Promotion Association CAS ; Royal Society Newton Advanced Fellowship[NAF\R2\192151] ; Research Grants Council of HKSAR[CityU 11212119] ; Centre for Applied Computing and Interactive Media (ACIM) of School of Creative Media, City University of Hong Kong
WOS研究方向Computer Science
语种英语
WOS记录号WOS:000830989200003
出版者ASSOC COMPUTING MACHINERY
源URL[http://119.78.100.204/handle/2XEOYT63/19436]  
专题中国科学院计算技术研究所期刊论文_英文
通讯作者Gao, Lin
作者单位1.Cardiff Univ, Sch Comp Sci & Informat, Cardiff, Wales
2.Chinese Acad Sci, Inst Comp Technol, Beijing Key Lab Mobile Comp & Pervas Device, Beijing, Peoples R China
3.Chinese Acad Sci, Inst Comp Technol, Beijing, Peoples R China
4.Univ Chinese Acad Sci, Beijing, Peoples R China
5.City Univ Hong Kong, Sch Creat Media, Hong Kong, Peoples R China
推荐引用方式
GB/T 7714		 Liu, Feng-Lin,Chen, Shu-Yu,Lai, Yu-Kun,et al. DeepFaceVideoEditing: Sketch-based Deep Editing of Face Videos[J]. ACM TRANSACTIONS ON GRAPHICS,2022,41(4):16.
APA Liu, Feng-Lin.,Chen, Shu-Yu.,Lai, Yu-Kun.,Li, Chunpeng.,Jiang, Yue-Ren.,...&Gao, Lin.(2022).DeepFaceVideoEditing: Sketch-based Deep Editing of Face Videos.ACM TRANSACTIONS ON GRAPHICS,41(4),16.
MLA Liu, Feng-Lin,et al."DeepFaceVideoEditing: Sketch-based Deep Editing of Face Videos".ACM TRANSACTIONS ON GRAPHICS 41.4(2022):16.

入库方式： OAI收割

来源：计算技术研究所

浏览0
下载0
收藏0
其他版本

除非特别说明，本系统中所有内容都受版权保护，并保留所有权利。