中国科学院机构知识库网格
Chinese Academy of Sciences Institutional Repositories Grid
ACMM: Aligned Cross-Modal Memory for Few-Shot Image and Sentence Matching

文献类型:会议论文

AuthorHuang, Yan; Wang, Liang
Issued Date2019-11
Conference Date2019.10.26-2019.11.2
Conference PlaceSeoul
KeywordImage And Sentence Matching
Volume0
Issue0
DOI0
Pages0
English Abstract

Image and sentence matching has drawn much attention
recently, but due to the lack of sufficient pairwise data for
training, most previous methods still cannot well associate
those challenging pairs of images and sentences containing
rarely appeared regions and words, i.e., few-shot content.
In this work, we study this challenging scenario as few-shot
image and sentence matching, and accordingly propose an
Aligned Cross-Modal Memory (ACMM) model to memorize
the rarely appeared content. Given a pair of image and sentence,
the model first includes an aligned memory controller
network to produce two sets of semantically-comparable interface
vectors through cross-modal alignment. Then the
interface vectors are used by modality-specific read and update
operations to alternatively interact with shared memory
items. The memory items persistently memorize crossmodal
shared semantic representations, which can be addressed
out to better enhance the representation of few-shot
content. We apply the proposed model to both conventional
and few-shot image and sentence matching tasks, and
demonstrate its effectiveness by achieving the state-of-theart
performance on two benchmark datasets.

Author of SourceKyoung Mu Lee
PublisherIEEE
Publish PlaceUSA
URL查看原文
源URL[http://ir.ia.ac.cn/handle/173211/25797]  
Collection自动化研究所_智能感知与计算研究中心
Affiliation中科院自动化所
Recommended Citation
GB/T 7714
Huang, Yan,Wang, Liang. ACMM: Aligned Cross-Modal Memory for Few-Shot Image and Sentence Matching[C]. 见:. Seoul. 2019.10.26-2019.11.2.

入库方式: OAI收割

来源:自动化研究所

浏览0
下载0
收藏0
其他版本

Items in the repository are protected by copyright, with all rights reserved, unless otherwise indicated.