Improving visual question answering using dropout and enhanced question encoder
文献类型:期刊论文
作者 | Fang, Zhiwei1,2; Liu, Jing1; Li, Yong3; Qiao, Yanyuan2; Lu, Hanqing1 |
刊名 | PATTERN RECOGNITION |
出版日期 | 2019-06-01 |
卷号 | 90期号:1页码:404-414 |
ISSN号 | 0031-3203 |
关键词 | Visual question answering Coherent dropout Siamese dropout Enhanced question encoder |
DOI | 10.1016/j.patcog.2019.01.038 |
英文摘要 | Using dropout in Visual Question Answering (VQA) is a common practice to prevent overfitting. However, the current way to use dropout in multi-path networks may cause two problems: the co-adaptations of neurons and the explosion of output variance. In this paper, we propose coherent dropout and siamese dropout mechanism to solve the two problems, respectively. Specifically, in coherent dropout, the relevant dropout layers in multiple paths are forced to work coherently to maximize the ability of preventing neuron co-adaptations. We show that the coherent dropout is simple in implementation but very effective to overcome overfitting. As for the explosion of output variance, we develop a siamese dropout mechanism to explicitly minimize the difference between the two output vectors produced from the same input data during training phase. Such mechanism can reduce the gap between training and inference phases and make the VQA model more robust. With the help of the two techniques, we further design an enhanced question encoder called Multi-path Stacked Residual RNNs which is deeper and wider and more powerful than current shallow question encoder. Extensive experiments are conducted to verify the effectiveness of coherent dropout, siamese dropout and the enhanced question encoder. And the results show that our methods can bring clear improvements to the state-of-the-art VQA models on VQA-vl and VQA-v2 datasets. (C) 2019 Elsevier Ltd. All rights reserved. |
WOS关键词 | NETWORKS |
资助项目 | National Natural Science Foundation of China[61872366] ; Beijing Municipal Natural Science Foundation[4192059] |
WOS研究方向 | Computer Science ; Engineering |
语种 | 英语 |
出版者 | ELSEVIER SCI LTD |
WOS记录号 | WOS:000463130400033 |
源URL | [http://ir.ia.ac.cn/handle/173211/23483] |
专题 | 自动化研究所_模式识别国家重点实验室_图像与视频分析团队 |
通讯作者 | Liu, Jing |
作者单位 | 1.Chinese Acad Sci, Inst Automat, Natl Lab Pattern Recognit, 95 Zhongguancun East Rd, Beijing 100190, Peoples R China 2.Univ Chinese Acad Sci, Beijing, Peoples R China 3.JD Com, Business Growth BU, Intelligent Advertising Lab, Beijing, Peoples R China |
推荐引用方式 GB/T 7714 | Fang, Zhiwei,Liu, Jing,Li, Yong,et al. Improving visual question answering using dropout and enhanced question encoder[J]. PATTERN RECOGNITION,2019,90(1):404-414. |
APA | Fang, Zhiwei,Liu, Jing,Li, Yong,Qiao, Yanyuan,&Lu, Hanqing.(2019).Improving visual question answering using dropout and enhanced question encoder.PATTERN RECOGNITION,90(1),404-414. |
MLA | Fang, Zhiwei,et al."Improving visual question answering using dropout and enhanced question encoder".PATTERN RECOGNITION 90.1(2019):404-414. |
入库方式: OAI收割
来源:自动化研究所
其他版本
除非特别说明,本系统中所有内容都受版权保护,并保留所有权利。