中国科学院机构知识库网格
Chinese Academy of Sciences Institutional Repositories Grid
首页 机构 成果 学者
  
  1. CAS IR Grid
  2. 计算技术研究所
  3. 中国科学院计算技术研究所
  4. 中国科学院计算技术研究所期刊论文
  5. 英文
Viewpoint-Adaptive Representation Disentanglement Network for Change Captioning

文献类型：期刊论文

作者Tu, Yunbin3; Li, Liang2,4; Su, Li3; Du, Junping5; Lu, Ke1,6; Huang, Qingming2,3
刊名IEEE TRANSACTIONS ON IMAGE PROCESSING
出版日期2023
卷号32页码:2620-2635
关键词Task analysis Image coding Adaptation models Encoding Computer science Transformers Semantics Change captioning representation disentanglement viewpoint-adaptive position-embedded representation learning
ISSN号1057-7149
DOI10.1109/TIP.2023.3268004
英文摘要Change captioning is to describe the fine-grained change between a pair of images. The pseudo changes caused by viewpoint changes are the most typical distractors in this task, because they lead to the feature perturbation and shift for the same objects and thus overwhelm the real change representation. In this paper, we propose a viewpoint-adaptive representation disentanglement network to distinguish real and pseudo changes, and explicitly capture the features of change to generate accurate captions. Concretely, a position-embedded representation learning is devised to facilitate the model in adapting to viewpoint changes via mining the intrinsic properties of two image representations and modeling their position information. To learn a reliable change representation for decoding into a natural language sentence, an unchanged representation disentanglement is designed to identify and disentangle the unchanged features between the two position-embedded representations. Extensive experiments show that the proposed method achieves the state-of-the-art performance on the four public datasets. The code is available at https://github.com/tuyunbin/VARD.
资助项目National Key Research and Development Program of China[2018YFE0303104] ; National Natural Science Foundation of China[62236008] ; National Natural Science Foundation of China[U21B2038] ; National Natural Science Foundation of China[61931008] ; National Natural Science Foundation of China[62032022] ; National Natural Science Foundation of China[62192784] ; National Natural Science Foundation of China[U22B2038] ; Youth Innovation Promotion Association of Chinese Academy of Sciences (CAS)[2020108] ; Key Research and Development Plan Project of Zhejiang Province[2023C01004]
WOS研究方向Computer Science ; Engineering
语种英语
WOS记录号WOS:000982402100011
出版者IEEE-INST ELECTRICAL ELECTRONICS ENGINEERS INC
源URL[http://119.78.100.204/handle/2XEOYT63/21413]  
专题中国科学院计算技术研究所期刊论文_英文
通讯作者Li, Liang; Su, Li
作者单位1.Univ Chinese Acad Sci, Sch Engn Sci, Beijing 101408, Peoples R China
2.Chinese Acad Sci, Inst Comp Technol, Beijing 100190, Peoples R China
3.Univ Chinese Acad Sci, Sch Comp Sci & Technol, Beijing 101408, Peoples R China
4.Hangzhou Dianzi Univ, Lishui Inst, Lishui 323000, Zhejiang, Peoples R China
5.Beijing Univ Posts & Telecommun, Sch Comp Sci & Technol, Beijing 100876, Peoples R China
6.Peng Cheng Lab, Shenzhen 518055, Guangdong, Peoples R China
推荐引用方式
GB/T 7714		 Tu, Yunbin,Li, Liang,Su, Li,et al. Viewpoint-Adaptive Representation Disentanglement Network for Change Captioning[J]. IEEE TRANSACTIONS ON IMAGE PROCESSING,2023,32:2620-2635.
APA Tu, Yunbin,Li, Liang,Su, Li,Du, Junping,Lu, Ke,&Huang, Qingming.(2023).Viewpoint-Adaptive Representation Disentanglement Network for Change Captioning.IEEE TRANSACTIONS ON IMAGE PROCESSING,32,2620-2635.
MLA Tu, Yunbin,et al."Viewpoint-Adaptive Representation Disentanglement Network for Change Captioning".IEEE TRANSACTIONS ON IMAGE PROCESSING 32(2023):2620-2635.

入库方式： OAI收割

来源：计算技术研究所

浏览0
下载0
收藏0
其他版本

除非特别说明，本系统中所有内容都受版权保护，并保留所有权利。