PSGAN plus plus : Robust Detail-Preserving Makeup Transfer and Removal
文献类型：期刊论文
| 作者 | Liu, Si2; Jiang, Wentao2; Gao, Chen2; He, Ran3 ; Feng, Jiashi1; Li, Bo2; Yan, Shuicheng4
|
| 刊名 | IEEE TRANSACTIONS ON PATTERN ANALYSIS AND MACHINE INTELLIGENCE
 |
| 出版日期 | 2022-11-01 |
| 卷号 | 44期号:11页码:8538-8551 |
| 关键词 | Faces Generative adversarial networks Task analysis Visualization Nose Image resolution Skin Makeup transfer makeup removal generative adversarial networks |
| ISSN号 | 0162-8828 |
| DOI | 10.1109/TPAMI.2021.3083484 |
| 通讯作者 | He, Ran(ran.he@ia.ac.cn) |
| 英文摘要 | In this paper, we address the makeup transfer and removal tasks simultaneously, which aim to transfer the makeup from a reference image to a source image and remove the makeup from the with-makeup image respectively. Existing methods have achieved much advancement in constrained scenarios, but it is still very challenging for them to transfer makeup between images with large pose and expression differences, or handle makeup details like blush on cheeks or highlight on the nose. In addition, they are hardly able to control the degree of makeup during transferring or to transfer a specified part in the input face. These defects limit the application of previous makeup transfer methods to real-world scenarios. In this work, we propose a Pose and expression robust Spatial-aware GAN (abbreviated as PSGAN++). PSGAN++ is capable of performing both detail-preserving makeup transfer and effective makeup removal. For makeup transfer, PSGAN++ uses a Makeup Distill Network (MDNet) to extract makeup information, which is embedded into spatial-aware makeup matrices. We also devise an Attentive Makeup Morphing (AMM) module that specifies how the makeup in the source image is morphed from the reference image, and a makeup detail loss to supervise the model within the selected makeup detail area. On the other hand, for makeup removal, PSGAN++ applies an Identity Distill Network (IDNet) to embed the identity information from with-makeup images into identity matrices. Finally, the obtained makeup/identity matrices are fed to a Style Transfer Network (STNet) that is able to edit the feature maps to achieve makeup transfer or removal. To evaluate the effectiveness of our PSGAN++, we collect a Makeup Transfer In the Wild (MT-Wild) dataset that contains images with diverse poses and expressions and a Makeup Transfer High-Resolution (MT-HR) dataset that contains high-resolution images. Experiments demonstrate that PSGAN++ not only achieves state-of-the-art results with fine makeup details even in cases of large pose/expression differences but also can perform partial or degree-controllable makeup transfer. Both the code and the newly collected datasets will be released at https://github.com/wtjiang98/PSGAN. |
| WOS关键词 | NETWORK |
| 资助项目 | National Natural Science Foundation of China[61876177] ; Beijing Natural Science Foundation[4202034] ; Beijing Natural Science Foundation[JQ18017] ; Zhejiang Laboratory[2019KD0AB04] |
| WOS研究方向 | Computer Science ; Engineering |
| 语种 | 英语 |
| WOS记录号 | WOS:000864325900088 |
| 出版者 | IEEE COMPUTER SOC |
| 资助机构 | National Natural Science Foundation of China ; Beijing Natural Science Foundation ; Zhejiang Laboratory |
| 源URL | [http://ir.ia.ac.cn/handle/173211/50296]  |
| 专题 | 自动化研究所_智能感知与计算研究中心 |
| 通讯作者 | He, Ran |
| 作者单位 | 1.Natl Univ Singapore, Singapore 119077, Singapore 2.Beihang Univ, Inst Artificial Intelligence, Beijing 100083, Peoples R China 3.Chinese Acad Sci, Inst Automat, Beijing 100049, Peoples R China 4.Sea AI Lab SAIL, Singapore 117576, Singapore |
| 推荐引用方式 GB/T 7714 | Liu, Si,Jiang, Wentao,Gao, Chen,et al. PSGAN plus plus : Robust Detail-Preserving Makeup Transfer and Removal[J]. IEEE TRANSACTIONS ON PATTERN ANALYSIS AND MACHINE INTELLIGENCE,2022,44(11):8538-8551. |
| APA | Liu, Si.,Jiang, Wentao.,Gao, Chen.,He, Ran.,Feng, Jiashi.,...&Yan, Shuicheng.(2022).PSGAN plus plus : Robust Detail-Preserving Makeup Transfer and Removal.IEEE TRANSACTIONS ON PATTERN ANALYSIS AND MACHINE INTELLIGENCE,44(11),8538-8551. |
| MLA | Liu, Si,et al."PSGAN plus plus : Robust Detail-Preserving Makeup Transfer and Removal".IEEE TRANSACTIONS ON PATTERN ANALYSIS AND MACHINE INTELLIGENCE 44.11(2022):8538-8551. |
入库方式： OAI收割
来源：自动化研究所
浏览0
下载0
收藏0
其他版本
除非特别说明，本系统中所有内容都受版权保护，并保留所有权利。