Associating Images with Sentences Using Recurrent Canonical Correlation Analysis
文献类型:期刊论文
作者 | Guo, Yawen2; Yuan, Hui1,3; Zhang, Kun3 |
刊名 | APPLIED SCIENCES-BASEL |
出版日期 | 2020-08-01 |
卷号 | 10期号:16页码:14 |
关键词 | image retrieval image and sentence matching canonical correlation analysis |
DOI | 10.3390/app10165516 |
通讯作者 | Guo, Yawen(yavinguo@outlook.com) |
英文摘要 | Associating images with sentences has drawn much attention recently. Existing methods commonly represent an image by indistinctively describing all its contents in an one-time static way, which ignores two facts that (1) the association analysis can only be effective for partial salient contents and the associated sentence, and (2) visual information acquisition is a dynamical rather than static process. To deal with this issue, we propose a recurrent canonical correlation analysis (RCCA) method for associating images with sentences. RCCA includes a contextual attention-based LSTM-RNN which can selectively attend to salient regions of an image at each time step, and then represent all the salient contents within a few steps. Different from existing attention-based models, our model focuses on the modelling of contextual visual attention mechanism for the task of association analysis. RCCA also includes a conventional LSTM-RNN for sentence representation learning. The resulting representations of images and sentences are fed into CCA to maximize linear correlation, where parameters of LSTM-RNNs and CCA are jointly learned. Due to the effective image representation learning, our model can well associate images with sentences with complex contents, and achieve better performance in terms of image annotation and retrieval. |
WOS研究方向 | Chemistry ; Engineering ; Materials Science ; Physics |
语种 | 英语 |
出版者 | MDPI |
WOS记录号 | WOS:000564772700001 |
源URL | [http://ir.ia.ac.cn/handle/173211/41516] |
专题 | 自动化研究所_复杂系统管理与控制国家重点实验室_先进机器人控制团队 |
通讯作者 | Guo, Yawen |
作者单位 | 1.Xiangtan Univ, Coll Automat & Elect Informat, Xiangtan 411105, Peoples R China 2.Chinese Acad Sci, Inst Biophys, Beijing 100190, Peoples R China 3.Chinese Acad Sci, Inst Automat, Beijing 100190, Peoples R China |
推荐引用方式 GB/T 7714 | Guo, Yawen,Yuan, Hui,Zhang, Kun. Associating Images with Sentences Using Recurrent Canonical Correlation Analysis[J]. APPLIED SCIENCES-BASEL,2020,10(16):14. |
APA | Guo, Yawen,Yuan, Hui,&Zhang, Kun.(2020).Associating Images with Sentences Using Recurrent Canonical Correlation Analysis.APPLIED SCIENCES-BASEL,10(16),14. |
MLA | Guo, Yawen,et al."Associating Images with Sentences Using Recurrent Canonical Correlation Analysis".APPLIED SCIENCES-BASEL 10.16(2020):14. |
入库方式: OAI收割
来源:自动化研究所
浏览0
下载0
收藏0
其他版本
除非特别说明,本系统中所有内容都受版权保护,并保留所有权利。