中国科学院机构知识库网格
Chinese Academy of Sciences Institutional Repositories Grid
Few-Shot Image and Sentence Matching via Gated Visual-Semantic Embedding

文献类型:会议论文

AuthorHuang, Yan; Long, Yang; Wang, Liang
Issued Date2019-01
Conference Date2019.1.27-2019.2.1
Conference PlaceHonolulu
KeywordImage And Sentence Matching
Volume33
Issue0
DOI0
Pages8489-8496
English Abstract

Although image and sentence matching has been widely studied,
its intrinsic few-shot problem is commonly ignored,
which has became a bottleneck for further performance improvement.
In this work, we focus on the challenging problem
of few-shot image and sentence matching, and propose a Gated
Visual-Semantic Embedding (GVSE) model to deal with
it. The model consists of three corporative modules in terms
of uncommon VSE, common VSE, and gated metric fusion.
The uncommon VSE exploits external auxiliary sources to
extract generic features for describing uncommon instances
and words in images and sentences, and then integrates them
by modeling their semantic relation to obtain global representations
for association analysis. To better model the most
common instances and words in rest content of images and
sentences, the common VSE learns their discriminative representations
directly from scratch. After obtaining two similarity
metrics from the two VSE modules, the gated metric
fusion module adaptively fuses them by automatically balancing
their relative importance. Based on the fused metric,
we performance extensive experiments in terms of few-shot
and conventional image and sentence matching, and demonstrate
the effectiveness of the proposed model by achieving
the state-of-the-art results on two public benchmark datasets.

Author of SourceZhihua Zhou
PublisherIEEE
Publish PlaceUSA
URL查看原文
源URL[http://ir.ia.ac.cn/handle/173211/25798]  
Collection自动化研究所_智能感知与计算研究中心
Affiliation中科院自动化所
Recommended Citation
GB/T 7714
Huang, Yan,Long, Yang,Wang, Liang. Few-Shot Image and Sentence Matching via Gated Visual-Semantic Embedding[C]. 见:. Honolulu. 2019.1.27-2019.2.1.

入库方式: OAI收割

来源:自动化研究所

浏览0
下载0
收藏0
其他版本

Items in the repository are protected by copyright, with all rights reserved, unless otherwise indicated.