中国科学院机构知识库网格
Chinese Academy of Sciences Institutional Repositories Grid
首页 机构 成果 学者
  
  1. CAS IR Grid
  2. 计算技术研究所
  3. 中国科学院计算技术研究所
  4. 中国科学院计算技术研究所期刊论文
  5. 英文
NeRFFaceShop: Learning a Photo-Realistic 3D-Aware Generative Model of Animatable and Relightable Heads From Large-Scale in-the-Wild Videos

文献类型：期刊论文

作者Jiang, Kaiwen1; Liu, Feng-Lin2,3; Chen, Shu-Yu2; Wan, Pengfei4; Zhang, Yuan4; Lai, Yu-Kun5; Fu, Hongbo6; Gao, Lin2,3
刊名IEEE TRANSACTIONS ON VISUALIZATION AND COMPUTER GRAPHICS
出版日期2025-10-01
卷号31期号:10页码:7938-7950
关键词Animation Lighting Head Three-dimensional displays Videos Training Computational modeling Solid modeling Aerospace electronics Rendering (computer graphics) Face animation face relighting volume disentangling neural radiance fields neural rendering
ISSN号1077-2626
DOI10.1109/TVCG.2025.3560869
英文摘要Animatable and relightable 3D facial generation has fundamental applications in computer vision and graphics. Although animation and relighting are highly correlated, previous methods usually address them separately. Effectively combining animation methods and relighting methods is nontrivial. In terms of explicit shading models, animatable methods cannot be easily extended to achieve realistic relighting results, such as shadow effects, due to prohibitive computational training costs. Regarding implicit lighting representations, current animatable methods cannot be incorporated due to their inharmonious animation representations, i.e., deforming spatial points. This paper, armed with a lightweight but effective lighting representation, presents a compatible animation representation to achieve a disentangled generative model of 3D animatable and relightable heads. Our represented animation allows for updating and control of realistic lighting effects. Due to the disentangled nature of our representations, we learn the animation and relighting from large-scale, in-the-wild videos instead of relying on a morphable model. We show that our method can synthesize geometrically consistent and detailed motion along with the disentangled control of lighting conditions. We further show that our method is still compatible with morphable models for driving generated avatars. Our method can also be extended to domains without video data by domain transfer to achieve a broader range of animatable and relightable head synthesis. We will release the code for reproducibility and facilitating future research.
资助项目Kuaishou Technology, Beijing Municipal Science and Technology Commission[Z231100005923031] ; National Natural Science Foundation of China[62322210] ; National Natural Science Foundation of China[62472407] ; Innovation Funding of ICT, CAS[E461020]
WOS研究方向Computer Science
语种英语
WOS记录号WOS:001566984900019
出版者IEEE COMPUTER SOC
源URL[http://119.78.100.204/handle/2XEOYT63/41728]  
专题中国科学院计算技术研究所期刊论文_英文
通讯作者Gao, Lin
作者单位1.Univ Calif San Diego, CSE Dept, La Jolla, CA 92093 USA
2.Chinese Acad Sci, Beijing Key Lab Mobile Comp & Pervas Device, Inst Comp Technol, Beijing 100864, Peoples R China
3.Univ Chinese Acad Sci, Beijing 101408, Peoples R China
4.Kuaishou Technol, Beijing 100085, Peoples R China
5.Cardiff Univ, Sch Comp Sci & Informat, Cardiff CF10 3AT, Wales
6.Hong Kong Univ Sci & Technol, Div Arts & Machine Creat, Hong Kong, Peoples R China
推荐引用方式
GB/T 7714		 Jiang, Kaiwen,Liu, Feng-Lin,Chen, Shu-Yu,et al. NeRFFaceShop: Learning a Photo-Realistic 3D-Aware Generative Model of Animatable and Relightable Heads From Large-Scale in-the-Wild Videos[J]. IEEE TRANSACTIONS ON VISUALIZATION AND COMPUTER GRAPHICS,2025,31(10):7938-7950.
APA Jiang, Kaiwen.,Liu, Feng-Lin.,Chen, Shu-Yu.,Wan, Pengfei.,Zhang, Yuan.,...&Gao, Lin.(2025).NeRFFaceShop: Learning a Photo-Realistic 3D-Aware Generative Model of Animatable and Relightable Heads From Large-Scale in-the-Wild Videos.IEEE TRANSACTIONS ON VISUALIZATION AND COMPUTER GRAPHICS,31(10),7938-7950.
MLA Jiang, Kaiwen,et al."NeRFFaceShop: Learning a Photo-Realistic 3D-Aware Generative Model of Animatable and Relightable Heads From Large-Scale in-the-Wild Videos".IEEE TRANSACTIONS ON VISUALIZATION AND COMPUTER GRAPHICS 31.10(2025):7938-7950.

入库方式： OAI收割

来源：计算技术研究所

浏览0
下载0
收藏0
其他版本

除非特别说明，本系统中所有内容都受版权保护，并保留所有权利。