Online Asymmetric Metric Learning With Multi-Layer Similarity Aggregation for Cross-Modal Retrieval
文献类型：期刊论文
| 作者 | Song, Guoli1,2; Wang, Shuhui3; Wu, Yiling1,2,3; Huang, Qingming1,2,3 |
| 刊名 | IEEE TRANSACTIONS ON IMAGE PROCESSING
 |
| 出版日期 | 2019-09-01 |
| 卷号 | 28期号:9页码:4299-4312 |
| 关键词 | Cross-modal retrieval asymmetric metric online learning multi-layer aggregation |
| ISSN号 | 1057-7149 |
| DOI | 10.1109/TIP.2019.2908774 |
| 英文摘要 | Cross-modal retrieval has attracted intensive attention in recent years, where a substantial yet challenging problem is how to measure the similarity between heterogeneous data modalities. Despite using modality-specific representation learning techniques, most existing shallow or deep models treat different modalities equally and neglect the intrinsic modality heterogeneity and information imbalance among images and texts. In this paper, we propose an online similarity function learning framework to learn the metric that can well reflect the cross-modal semantic relation. Considering that multiple CNN feature layers naturally represent visual information from low-level visual patterns to high-level semantic abstraction, we propose a new asymmetric image-text similarity formulation which aggregates the layer-wise visual-textual similarities parameterized by different bilinear parameter matrices. To effectively learn the aggregated similarity function, we develop three different similarity combination strategies, i.e., average kernel, multiple kernel learning, and layer gating. The former two kernel-based strategies assign uniform weights on different layers to all data pairs; the latter works on the original feature representation and assigns instance-aware weights on different layers to different data pairs, and they are all learned by preserving the bidirectional relative similarity expressed by a large number of cross-modal training triplets. The experiments conducted on three public datasets well demonstrate the effectiveness of our methods. |
| 资助项目 | National Natural Science Foundation of China[61672497] ; National Natural Science Foundation of China[61620106009] ; National Natural Science Foundation of China[U1636214] ; National Natural Science Foundation of China[61836002] ; National Basic Research Program of China (973 Program)[2015CB351800] ; China Postdoctoral Science Foundation[119103S291] ; Key Research Program of Frontier Sciences of CAS[QYZDJ-SSW-SYS013] |
| WOS研究方向 | Computer Science ; Engineering |
| 语种 | 英语 |
| WOS记录号 | WOS:000473641100009 |
| 出版者 | IEEE-INST ELECTRICAL ELECTRONICS ENGINEERS INC |
| 源URL | [http://119.78.100.204/handle/2XEOYT63/4317]  |
| 专题 | 中国科学院计算技术研究所期刊论文_英文 |
| 通讯作者 | Wang, Shuhui |
| 作者单位 | 1.UCAS, Sch Comp Sci & Technol, Beijing 101408, Peoples R China 2.UCAS, Key Lab Big Data Min & Knowledge Management, Beijing 101408, Peoples R China 3.Chinese Acad Sci, Inst Comp Technol, Key Lab Intelligent Informat Proc, Beijing 100190, Peoples R China |
| 推荐引用方式 GB/T 7714 | Song, Guoli,Wang, Shuhui,Wu, Yiling,et al. Online Asymmetric Metric Learning With Multi-Layer Similarity Aggregation for Cross-Modal Retrieval[J]. IEEE TRANSACTIONS ON IMAGE PROCESSING,2019,28(9):4299-4312. |
| APA | Song, Guoli,Wang, Shuhui,Wu, Yiling,&Huang, Qingming.(2019).Online Asymmetric Metric Learning With Multi-Layer Similarity Aggregation for Cross-Modal Retrieval.IEEE TRANSACTIONS ON IMAGE PROCESSING,28(9),4299-4312. |
| MLA | Song, Guoli,et al."Online Asymmetric Metric Learning With Multi-Layer Similarity Aggregation for Cross-Modal Retrieval".IEEE TRANSACTIONS ON IMAGE PROCESSING 28.9(2019):4299-4312. |
入库方式： OAI收割
来源：计算技术研究所
浏览0
下载0
收藏0
其他版本
除非特别说明，本系统中所有内容都受版权保护，并保留所有权利。