中国科学院机构知识库网格
Chinese Academy of Sciences Institutional Repositories Grid
首页 机构 成果 学者
  
  1. CAS IR Grid
  2. 计算技术研究所
  3. 中国科学院计算技术研究所
  4. 中国科学院计算技术研究所期刊论文
  5. 英文
Learning to Recognize Visual Concepts for Visual Question Answering With Structural Label Space

文献类型：期刊论文

作者Gao, Difei1,2; Wang, Ruiping1,2; Shan, Shiguang1,2; Chen, Xilin1,2
刊名IEEE JOURNAL OF SELECTED TOPICS IN SIGNAL PROCESSING
出版日期2020-03-01
卷号14期号:3页码:494-505
关键词Visualization Semantics Grounding Knowledge discovery Sports Task analysis Image recognition Visual question answering visual concept recognition structural label space
ISSN号1932-4553
DOI10.1109/JSTSP.2020.2989701
英文摘要Solving visual question answering (VQA) task requires recognizing many diverse visual concepts as the answer. These visual concepts contain rich structural semantic meanings, e.g., some concepts in VQA are highly related (e.g., red & blue), some of them are less relevant (e.g., red & standing). It is very natural for humans to efficiently learn concepts by utilizing their semantic meanings to concentrate on distinguishing relevant concepts and eliminate the disturbance of irrelevant concepts. However, previous works usually use a simple MLP to output visual concept as the answer in a flat label space that treats all labels equally, causing limitations in representing and using the semantic meanings of labels. To address this issue, we propose a novel visual recognition module named Dynamic Concept Recognizer (DCR), which is easy to be plugged in an attention-based VQA model, to utilize the semantics of the labels in answer prediction. Concretely, we introduce two key features in DCR: 1) a novel structural label space to depict the difference of semantics between concepts, where the labels in new label space are assigned to different groups according to their meanings. This type of semantic information helps decompose the visual recognizer in VQA into multiple specialized sub-recognizers to improve the capacity and efficiency of the recognizer. 2) A feature attention mechanism to capture the similarity between relevant groups of concepts, e.g., human-related group "chef, waiter" is more related to "swimming, running, etc." than scene related group "sunny, rainy, etc.". This type of semantic information helps sub-recognizers for relevant groups to adaptively share part of modules and to share the knowledge between relevant sub-recognizers to facilitate the learning procedure. Extensive experiments on several datasets have shown that the proposed structural label space and DCR module can efficiently learn the visual concept recognition and benefit the performance of the VQA model.
资助项目Natural Science Foundation of China[U19B2036] ; Natural Science Foundation of China[61922080] ; Natural Science Foundation of China[61772500] ; CAS Frontier Science Key Research Project[QYZDJ-SSWJSC009]
WOS研究方向Engineering
语种英语
WOS记录号WOS:000543960100003
出版者IEEE-INST ELECTRICAL ELECTRONICS ENGINEERS INC
源URL[http://119.78.100.204/handle/2XEOYT63/15035]  
专题中国科学院计算技术研究所期刊论文_英文
通讯作者Chen, Xilin
作者单位1.Chinese Acad Sci, Chinese Acad Sci, Inst Comp Technol, Key Lab Intelligent Informat Proc, Beijing 100190, Peoples R China
2.Univ Chinese Acad Sci, Beijing 100049, Peoples R China
推荐引用方式
GB/T 7714		 Gao, Difei,Wang, Ruiping,Shan, Shiguang,et al. Learning to Recognize Visual Concepts for Visual Question Answering With Structural Label Space[J]. IEEE JOURNAL OF SELECTED TOPICS IN SIGNAL PROCESSING,2020,14(3):494-505.
APA Gao, Difei,Wang, Ruiping,Shan, Shiguang,&Chen, Xilin.(2020).Learning to Recognize Visual Concepts for Visual Question Answering With Structural Label Space.IEEE JOURNAL OF SELECTED TOPICS IN SIGNAL PROCESSING,14(3),494-505.
MLA Gao, Difei,et al."Learning to Recognize Visual Concepts for Visual Question Answering With Structural Label Space".IEEE JOURNAL OF SELECTED TOPICS IN SIGNAL PROCESSING 14.3(2020):494-505.

入库方式： OAI收割

来源：计算技术研究所

浏览0
下载0
收藏0
其他版本

除非特别说明，本系统中所有内容都受版权保护，并保留所有权利。