中国科学院机构知识库网格
Chinese Academy of Sciences Institutional Repositories Grid
首页 机构 成果 学者
  
  1. CAS IR Grid
  2. 计算技术研究所
  3. 中国科学院计算技术研究所
  4. 中国科学院计算技术研究所期刊论文
  5. 英文
Reference-Based Deep Line Art Video Colorization

文献类型：期刊论文

作者Shi, Min2; Zhang, Jia-Qi1; Chen, Shu-Yu3,5; Gao, Lin3,5; Lai, Yu-Kun4; Zhang, Fang-Lue6
刊名IEEE TRANSACTIONS ON VISUALIZATION AND COMPUTER GRAPHICS
出版日期2023-06-01
卷号29期号:6页码:2965-2979
关键词Image color analysis Art Animation Feature extraction Three-dimensional displays Transforms Color Line art colorization color transform temporal coherence few shot learning
ISSN号1077-2626
DOI10.1109/TVCG.2022.3146000
英文摘要Coloring line art images based on the colors of reference images is a crucial stage in animation production, which is time-consuming and tedious. This paper proposes a deep architecture to automatically color line art videos with the same color style as the given reference images. Our framework consists of a color transform network and a temporal refinement network based on 3U-net. The color transform network takes the target line art images as well as the line art and color images of the reference images as input and generates corresponding target color images. To cope with the large differences between each target line art image and the reference color images, we propose a distance attention layer that utilizes non-local similarity matching to determine the region correspondences between the target image and the reference images and transforms the local color information from the references to the target. To ensure global color style consistency, we further incorporate Adaptive Instance Normalization (AdaIN) with the transformation parameters obtained from a multiple-layer AdaIN that describes the global color style of the references extracted by an embedder network. The temporal refinement network learns spatiotemporal features through 3D convolutions to ensure the temporal color consistency of the results. Our model can achieve even better coloring results by fine-tuning the parameters with only a small number of samples when dealing with an animation of a new style. To evaluate our method, we build a line art coloring dataset. Experiments show that our method achieves the best performance on line art video coloring compared to the current state-of-the-art methods.
资助项目National Natural Science Foundation of China[61972379] ; National Natural Science Foundation of China[62102403] ; National Natural Science Foundation of China[61872440] ; Science and Technology Service Network Initiative, Chinese Academy of Sciences[KFJ-STS-QYZD-2021-11-001] ; Royal Society Newton Advanced Fellowship[NAF\R2\192151] ; Royal Society[IES\R1\180126] ; Youth Innovation Promotion Association CAS ; Marsden Fund Council[MFP-20-VUW-180]
WOS研究方向Computer Science
语种英语
WOS记录号WOS:000981880500011
出版者IEEE COMPUTER SOC
源URL[http://119.78.100.204/handle/2XEOYT63/21425]  
专题中国科学院计算技术研究所期刊论文_英文
通讯作者Gao, Lin
作者单位1.Beihang Univ, Beijing 100191, Peoples R China
2.North China Elect Power Univ, Beijing 102206, Peoples R China
3.Chinese Acad Sci, Inst Comp Technol, Beijing Key Lab Mobile Comp & Pervas Device, Beijing 100190, Peoples R China
4.Cardiff Univ, Sch Comp Sci & Informat, Cardiff CF10 3AT, Wales
5.Univ Chinese Acad Sci, Beijing 100190, Peoples R China
6.Victoria Univ Wellington, Sch Engn & Comp Sci, Wellington 6012, New Zealand
推荐引用方式
GB/T 7714		 Shi, Min,Zhang, Jia-Qi,Chen, Shu-Yu,et al. Reference-Based Deep Line Art Video Colorization[J]. IEEE TRANSACTIONS ON VISUALIZATION AND COMPUTER GRAPHICS,2023,29(6):2965-2979.
APA Shi, Min,Zhang, Jia-Qi,Chen, Shu-Yu,Gao, Lin,Lai, Yu-Kun,&Zhang, Fang-Lue.(2023).Reference-Based Deep Line Art Video Colorization.IEEE TRANSACTIONS ON VISUALIZATION AND COMPUTER GRAPHICS,29(6),2965-2979.
MLA Shi, Min,et al."Reference-Based Deep Line Art Video Colorization".IEEE TRANSACTIONS ON VISUALIZATION AND COMPUTER GRAPHICS 29.6(2023):2965-2979.

入库方式： OAI收割

来源：计算技术研究所

浏览0
下载0
收藏0
其他版本

除非特别说明，本系统中所有内容都受版权保护，并保留所有权利。