Bidirectional difference locating and semantic consistency reasoning for change captioning
文献类型：期刊论文
| 作者 | Sun, Yaoqi1; Li, Liang2; Yao, Tingting1; Lu, Tongyv1; Zheng, Bolun1; Yan, Chenggang1; Zhang, Hua1; Bao, Yongjun3; Ding, Guiguang4; Slabaugh, Gregory5 |
| 刊名 | INTERNATIONAL JOURNAL OF INTELLIGENT SYSTEMS
 |
| 出版日期 | 2022-01-19 |
| 页码 | 19 |
| ISSN号 | 0884-8173 |
| 关键词 | change captioning semantic consistency reasoning spatial-channel attention |
| DOI | 10.1002/int.22821 |
| 英文摘要 | Change captioning is an emerging task to describe the changes between a pair of images. The difficulty in this task is to discover the differences between the two images. Recently, some methods have been proposed to address this problem. However, they all employ unidirectional difference localization to identify the changes. This can lead to ambiguity about the nature of the changes. Instead, we propose a framework with bidirectional difference localization and semantic consistency reasoning to describe the image changes. First, we locate the changes in the two images by capturing bidirectional differences. Then we design a decoder with spatial-channel attention to generate the change caption. Finally, we introduce semantic consistency reasoning to constrain our bidirectional difference localization module and spatial-channel attention module. Extensive experiments on three public data sets show that the performance of our proposed model outperforms the state-of-the-art change captioning models by a large margin. |
| WOS研究方向 | Computer Science |
| 语种 | 英语 |
| 出版者 | WILEY |
| WOS记录号 | WOS:000744353300001 |
| 源URL | [http://119.78.100.204/handle/2XEOYT63/18301]  |
| 专题 | 中国科学院计算技术研究所期刊论文_英文 |
| 通讯作者 | Li, Liang |
| 作者单位 | 1.Hangzhou Dianzi Univ, Coll Comp Sci & Technol, Sch Automat, Hangzhou, Peoples R China 2.Chinese Acad Sci, Inst Comp Technol, Beijing 100086, Peoples R China 3.JD Com, Data & Intelligence Dept, Beijing, Peoples R China 4.Tsinghua Univ, Sch Software, Beijing, Peoples R China 5.Queen Mary Univ London, Digital Environm Res Inst DERI, London, England |
| 推荐引用方式 GB/T 7714 | Sun, Yaoqi,Li, Liang,Yao, Tingting,et al. Bidirectional difference locating and semantic consistency reasoning for change captioning[J]. INTERNATIONAL JOURNAL OF INTELLIGENT SYSTEMS,2022:19. |
| APA | Sun, Yaoqi.,Li, Liang.,Yao, Tingting.,Lu, Tongyv.,Zheng, Bolun.,...&Slabaugh, Gregory.(2022).Bidirectional difference locating and semantic consistency reasoning for change captioning.INTERNATIONAL JOURNAL OF INTELLIGENT SYSTEMS,19. |
| MLA | Sun, Yaoqi,et al."Bidirectional difference locating and semantic consistency reasoning for change captioning".INTERNATIONAL JOURNAL OF INTELLIGENT SYSTEMS (2022):19. |
入库方式： OAI收割
来源：计算技术研究所
浏览0
下载0
收藏0
其他版本
除非特别说明，本系统中所有内容都受版权保护，并保留所有权利。