SAC-GAN: Structure-Aware Image Composition
文献类型：期刊论文
| 作者 | Zhou, Hang1; Ma, Rui2,3; Zhang, Ling-Xiao4; Gao, Lin4; Mahdavi-Amiri, Ali1; Zhang, Hao1 |
| 刊名 | IEEE TRANSACTIONS ON VISUALIZATION AND COMPUTER GRAPHICS
 |
| 出版日期 | 2024-07-01 |
| 卷号 | 30期号:7页码:3151-3165 |
| 关键词 | Layout Transforms Semantics Three-dimensional displays Image edge detection Codes Coherence Structure-aware image composition self-supervision GANs |
| ISSN号 | 1077-2626 |
| DOI | 10.1109/TVCG.2022.3226689 |
| 英文摘要 | We introduce an end-to-end learning framework for image-to-image composition, aiming to plausibly compose an object represented as a cropped patch from an object image into a background scene image. As our approach emphasizes more on semantic and structural coherence of the composed images, rather than their pixel-level RGB accuracies, we tailor the input and output of our network with structure-aware features and design our network losses accordingly, with ground truth established in a self-supervised setting through the object cropping. Specifically, our network takes the semantic layout features from the input scene image, features encoded from the edges and silhouette in the input object patch, as well as a latent code as inputs, and generates a 2D spatial affine transform defining the translation and scaling of the object patch. The learned parameters are further fed into a differentiable spatial transformer network to transform the object patch into the target image, where our model is trained adversarially using an affine transform discriminator and a layout discriminator. We evaluate our network, coined SAC-GAN, for various image composition scenarios in terms of quality, composability, and generalizability of the composite images. Comparisons are made to state-of-the-art alternatives, including Instance Insertion, ST-GAN, CompGAN and PlaceNet, confirming superiority of our method. |
| 资助项目 | NSERC Discovery[611370] ; National Natural Science Funds of China[62202199] |
| WOS研究方向 | Computer Science |
| 语种 | 英语 |
| WOS记录号 | WOS:001258936700035 |
| 出版者 | IEEE COMPUTER SOC |
| 源URL | [http://119.78.100.204/handle/2XEOYT63/39834]  |
| 专题 | 中国科学院计算技术研究所期刊论文_英文 |
| 通讯作者 | Ma, Rui |
| 作者单位 | 1.Simon Fraser Univ, Sch Comp Sci, Burnaby, BC V5A 1S6, Canada 2.Jilin Univ, Sch Artificial Intelligence, Changchun 130012, Peoples R China 3.Minist Educ, Engn Res Ctr Knowledge Driven Human Machine Intell, Changchun 130012, Peoples R China 4.Chinese Acad Sci, Inst Comp Technol, Beijing 100045, Peoples R China |
| 推荐引用方式 GB/T 7714 | Zhou, Hang,Ma, Rui,Zhang, Ling-Xiao,et al. SAC-GAN: Structure-Aware Image Composition[J]. IEEE TRANSACTIONS ON VISUALIZATION AND COMPUTER GRAPHICS,2024,30(7):3151-3165. |
| APA | Zhou, Hang,Ma, Rui,Zhang, Ling-Xiao,Gao, Lin,Mahdavi-Amiri, Ali,&Zhang, Hao.(2024).SAC-GAN: Structure-Aware Image Composition.IEEE TRANSACTIONS ON VISUALIZATION AND COMPUTER GRAPHICS,30(7),3151-3165. |
| MLA | Zhou, Hang,et al."SAC-GAN: Structure-Aware Image Composition".IEEE TRANSACTIONS ON VISUALIZATION AND COMPUTER GRAPHICS 30.7(2024):3151-3165. |
入库方式： OAI收割
来源：计算技术研究所
浏览0
下载0
收藏0
其他版本
除非特别说明，本系统中所有内容都受版权保护，并保留所有权利。