Biphasic Face Photo-Sketch Synthesis via Semantic-Driven Generative Adversarial Network With Graph Representation Learning
文献类型:期刊论文
作者 | Qi, Xingqun1,2,3; Sun, Muyi4,5; Wang, Zijian6; Liu, Jiaming7; Li, Qi4; Zhao, Fang8; Zhang, Shanghang7; Shan, Caifeng8,9 |
刊名 | IEEE TRANSACTIONS ON NEURAL NETWORKS AND LEARNING SYSTEMS |
出版日期 | 2023-12-19 |
页码 | 14 |
ISSN号 | 2162-237X |
关键词 | Face photo-sketch synthesis generative adversarial network graph representation learning intraclass and interclass iterative cycle training (ICT) |
DOI | 10.1109/TNNLS.2023.3341246 |
英文摘要 | Biphasic face photo-sketch synthesis has significant practical value in wide-ranging fields such as digital entertainment and law enforcement. Previous approaches directly generate the photo-sketch in a global view, they always suffer from the low quality of sketches and complex photograph variations, leading to unnatural and low-fidelity results. In this article, we propose a novel semantic-driven generative adversarial network to address the above issues, cooperating with graph representation learning. Considering that human faces have distinct spatial structures, we first inject class-wise semantic layouts into the generator to provide style-based spatial information for synthesized face photographs and sketches. In addition, to enhance the authenticity of details in generated faces, we construct two types of representational graphs via semantic parsing maps upon input faces, dubbed the intraclass semantic graph (IASG) and the interclass structure graph (IRSG). Specifically, the IASG effectively models the intraclass semantic correlations of each facial semantic component, thus producing realistic facial details. To preserve the generated faces being more structure-coordinated, the IRSG models interclass structural relations among every facial component by graph representation learning. To further enhance the perceptual quality of synthesized images, we present a biphasic interactive cycle training strategy by fully taking advantage of the multilevel feature consistency between the photograph and sketch. Extensive experiments demonstrate that our method outperforms the state-of-the-art competitors on the CUHK Face Sketch (CUFS) and CUHK Face Sketch FERET (CUFSF) datasets. |
资助项目 | Talent Introduction Program for Youth Innovation Teams of Shandong Province |
WOS研究方向 | Computer Science ; Engineering |
语种 | 英语 |
出版者 | IEEE-INST ELECTRICAL ELECTRONICS ENGINEERS INC |
WOS记录号 | WOS:001130341400001 |
资助机构 | Talent Introduction Program for Youth Innovation Teams of Shandong Province |
源URL | [http://ir.ia.ac.cn/handle/173211/54925] |
专题 | 多模态人工智能系统全国重点实验室 |
通讯作者 | Shan, Caifeng |
作者单位 | 1.Chinese Acad Sci, Inst Automat, Beijing 100190, Peoples R China 2.Peking Univ, Sch Comp Sci, Beijing 100871, Peoples R China 3.Hong Kong Univ Sci & Technol, Acad Interdisciplinary Studies, Hong Kong, Peoples R China 4.Chinese Acad Sci, Inst Automat, NLPR, CRIPAC, Beijing 100190, Peoples R China 5.Beijing Univ Posts & Telecommun, Sch Artificial Intelligence, Beijing 100876, Peoples R China 6.Univ Sydney, Sch Comp Sci, Sydney, NSW 2008, Australia 7.Peking Univ, Sch Comp Sci, Natl Key Lab Multimedia Informat Proc, Beijing 100871, Peoples R China 8.Nanjing Univ, Sch Intelligence Sci & Technol, Nanjing 210023, Peoples R China 9.Shandong Univ Sci & Technol, Coll Elect Engn & Automat, Qingdao 266590, Peoples R China |
推荐引用方式 GB/T 7714 | Qi, Xingqun,Sun, Muyi,Wang, Zijian,et al. Biphasic Face Photo-Sketch Synthesis via Semantic-Driven Generative Adversarial Network With Graph Representation Learning[J]. IEEE TRANSACTIONS ON NEURAL NETWORKS AND LEARNING SYSTEMS,2023:14. |
APA | Qi, Xingqun.,Sun, Muyi.,Wang, Zijian.,Liu, Jiaming.,Li, Qi.,...&Shan, Caifeng.(2023).Biphasic Face Photo-Sketch Synthesis via Semantic-Driven Generative Adversarial Network With Graph Representation Learning.IEEE TRANSACTIONS ON NEURAL NETWORKS AND LEARNING SYSTEMS,14. |
MLA | Qi, Xingqun,et al."Biphasic Face Photo-Sketch Synthesis via Semantic-Driven Generative Adversarial Network With Graph Representation Learning".IEEE TRANSACTIONS ON NEURAL NETWORKS AND LEARNING SYSTEMS (2023):14. |
入库方式: OAI收割
来源:自动化研究所
其他版本
除非特别说明,本系统中所有内容都受版权保护,并保留所有权利。