Re-ranking image-text matching by adaptive metric fusion
文献类型：期刊论文
| 作者 | Niu, Kai2,3 ; Huang, Yan2,3; Wang, Liang1,2,3,4
|
| 刊名 | PATTERN RECOGNITION
 |
| 出版日期 | 2020-08-01 |
| 卷号 | 104页码:13 |
| ISSN号 | 0031-3203 |
| 关键词 | Image-text matching Re-ranking method Adaptive metric fusion |
| DOI | 10.1016/j.patcog.2020.107351 |
| 通讯作者 | Wang, Liang(wangliang@nlpr.ia.ac.cn) |
| 英文摘要 | Image-text matching has drawn much attention recently with the rapid growth of multi-modal data. Many effective approaches have been proposed to solve this challenging problem, but limited effort has been devoted to re-ranking methods. Compared with the uni-modal re-ranking methods, modality heterogeneity is the major difficulty when designing a re-ranking method in the cross-modal field, which mainly lies in two aspects of different visual and textual feature spaces and different distributions in inverse directions. In this paper, we propose a heuristic re-ranking method called Adaptive Metric Fusion (AMF) for image-text matching. The method can obtain a better metric by adaptively fusing metrics based on two modules: 1) Cross-modal Reciprocal Encoding, which considers ranks in inverse directions to comprehensively evaluate a metric. The sentence retrieval and image retrieval have different distribution characteristics and galleries in different modalities, thus it is necessary to exploit them simultaneously for appropriate metric fusion. 2) Query Replacement Gap, which quantifies the gap between cross-modal and uni-modal similarities to alleviate the influence of different visual and textual feature spaces on the fused metric. The proposed re-ranking method can be implemented in an unsupervised way without requiring any human interaction or annotated data, and can be easily applied to any initial ranking result. Extensive experiments and analysis validate the effectiveness of our method on the large-scale MS-COCO and Flickr30K datasets. (C) 2020 Elsevier Ltd. All rights reserved. |
| 资助项目 | National Key Research and Development Program of China[2016YFB1001000] ; Key Research Program of Frontier Sciences, CAS[ZDBS-LY-JSC032] ; National Natural Science Foundation of China[61525306] ; National Natural Science Foundation of China[61633021] ; National Natural Science Foundation of China[61721004] ; National Natural Science Foundation of China[61806194] ; National Natural Science Foundation of China[U1803261] ; National Natural Science Foundation of China[61976132] ; Shandong Provincial Key Research and Development Program[2019JZZY010119] ; Shandong Provincial Key Research and Development Program[HW2019SOW01] ; CAS-AIR |
| WOS研究方向 | Computer Science ; Engineering |
| 语种 | 英语 |
| 出版者 | ELSEVIER SCI LTD |
| WOS记录号 | WOS:000532701300022 |
| 资助机构 | National Key Research and Development Program of China ; Key Research Program of Frontier Sciences, CAS ; National Natural Science Foundation of China ; Shandong Provincial Key Research and Development Program ; CAS-AIR |
| 源URL | [http://ir.ia.ac.cn/handle/173211/39454]  |
| 专题 | 自动化研究所_智能感知与计算研究中心 |
| 通讯作者 | Wang, Liang |
| 作者单位 | 1.Chinese Acad Sci CAS AIR, Artificial Intelligence Res, Qingdao 266200, Peoples R China 2.Univ Chinese Acad Sci UCAS, Beijing 100049, Peoples R China 3.Chinese Acad Sci CASIA, Ctr Res Intelligent Percept & Comp CRIPAC, Inst Automat, Natl Lab Pattern Recognit NLPR, Beijing 100190, Peoples R China 4.Chinese Acad Sci CASIA, Ctr Excellence Brain Sci & Intelligence Technol C, Inst Automat, Beijing 100190, Peoples R China |
| 推荐引用方式 GB/T 7714 | Niu, Kai,Huang, Yan,Wang, Liang. Re-ranking image-text matching by adaptive metric fusion[J]. PATTERN RECOGNITION,2020,104:13. |
| APA | Niu, Kai,Huang, Yan,&Wang, Liang.(2020).Re-ranking image-text matching by adaptive metric fusion.PATTERN RECOGNITION,104,13. |
| MLA | Niu, Kai,et al."Re-ranking image-text matching by adaptive metric fusion".PATTERN RECOGNITION 104(2020):13. |
入库方式： OAI收割
来源：自动化研究所
浏览0
下载0
收藏0
其他版本
除非特别说明，本系统中所有内容都受版权保护，并保留所有权利。