中国科学院机构知识库网格
Chinese Academy of Sciences Institutional Repositories Grid
首页 机构 成果 学者
  
  1. CAS IR Grid
  2. 计算技术研究所
  3. 中国科学院计算技术研究所
  4. 中国科学院计算技术研究所期刊论文
  5. 英文
Identity-Aware and Shape-Aware Propagation of Face Editing in Videos

文献类型：期刊论文

作者Jiang, Yue-Ren1,2; Chen, Shu-Yu1; Fu, Hongbo3; Gao, Lin1,2
刊名IEEE TRANSACTIONS ON VISUALIZATION AND COMPUTER GRAPHICS
出版日期2024-07-01
卷号30期号:7页码:3444-3456
关键词Videos Shape Faces Three-dimensional displays Codes Aerospace electronics Semantics Editing propagation face editing video editing
ISSN号1077-2626
DOI10.1109/TVCG.2023.3235364
英文摘要The development of deep generative models has inspired various facial image editing methods, but many of them are difficult to be directly applied to video editing due to various challenges ranging from imposing 3D constraints, preserving identity consistency, ensuring temporal coherence, etc. To address these challenges, we propose a new framework operating on the StyleGAN2 latent space for identity-aware and shape-aware edit propagation on face videos. In order to reduce the difficulties of maintaining the identity, keeping the original 3D motion, and avoiding shape distortions, we disentangle the StyleGAN2 latent vectors of human face video frames to decouple the appearance, shape, expression, and motion from identity. An edit encoding module is used to map a sequence of image frames to continuous latent codes with 3D parametric control and is trained in a self-supervised manner with identity loss and triple shape losses. Our model supports propagation of edits in various forms: I. direct appearance editing on a specific keyframe, II. implicit editing of face shape via a given reference image, and III. existing latent-based semantic edits. Experiments show that our method works well for various forms of videos in the wild and outperforms an animation-based approach and the recent deep generative techniques.
资助项目National Natural Science Foundation of China[62102403] ; National Natural Science Foundation of China[61872440] ; Beijing Municipal Natural Science Foundation for Distinguished Young Scholars[JQ21013] ; China Postdoctoral Science Foundation[2022M713205] ; Open Research Projects of Zhejiang Lab[2021KE0AB06] ; Youth Innovation Promotion Association CAS ; Centre for Applied Computing and Interactive Media (ACIM) of School of Creative Media, CityU
WOS研究方向Computer Science
语种英语
WOS记录号WOS:001258936700019
出版者IEEE COMPUTER SOC
源URL[http://119.78.100.204/handle/2XEOYT63/39836]  
专题中国科学院计算技术研究所期刊论文_英文
通讯作者Gao, Lin
作者单位1.Chinese Acad Sci, Inst Comp Technol, Beijing Key Lab Mobile Comp & Pervas Device, Beijing 100045, Peoples R China
2.Univ Chinese Acad Sci, Beijing 101408, Peoples R China
3.City Univ Hong Kong, Sch Creat Media, Hong Kong, Peoples R China
推荐引用方式
GB/T 7714		 Jiang, Yue-Ren,Chen, Shu-Yu,Fu, Hongbo,et al. Identity-Aware and Shape-Aware Propagation of Face Editing in Videos[J]. IEEE TRANSACTIONS ON VISUALIZATION AND COMPUTER GRAPHICS,2024,30(7):3444-3456.
APA Jiang, Yue-Ren,Chen, Shu-Yu,Fu, Hongbo,&Gao, Lin.(2024).Identity-Aware and Shape-Aware Propagation of Face Editing in Videos.IEEE TRANSACTIONS ON VISUALIZATION AND COMPUTER GRAPHICS,30(7),3444-3456.
MLA Jiang, Yue-Ren,et al."Identity-Aware and Shape-Aware Propagation of Face Editing in Videos".IEEE TRANSACTIONS ON VISUALIZATION AND COMPUTER GRAPHICS 30.7(2024):3444-3456.

入库方式： OAI收割

来源：计算技术研究所

浏览0
下载0
收藏0
其他版本

除非特别说明，本系统中所有内容都受版权保护，并保留所有权利。