Object-difference drived graph convolutional networks for visual question answering
文献类型：期刊论文
| 作者 | Zhu, Xi4,5; Mao, Zhendong1; Chen, Zhineng6 ; Li, Yangyang2; Wang, Zhaohui4,5; Wang, Bin3
|
| 刊名 | MULTIMEDIA TOOLS AND APPLICATIONS
 |
| 出版日期 | 2020-03-20 |
| 页码 | 19 |
| 关键词 | Visual question answering Graph convolutional networks Object-difference |
| ISSN号 | 1380-7501 |
| DOI | 10.1007/s11042-020-08790-0 |
| 通讯作者 | Mao, Zhendong(maozhendong2008@gmail.com) |
| 英文摘要 | Visual Question Answering(VQA), an important task to evaluate the cross-modal understanding capability of an Artificial Intelligence model, has been a hot research topic in both computer vision and natural language processing communities. Recently, graph-based models have received growing interest in VQA, for its potential of modeling the relationships between objects as well as its formidable interpretability. Nonetheless, those solutions mainly define the similarity between objects as their semantical relationships, while largely ignoring the critical point that the difference between objects can provide more information for establishing the relationship between nodes in the graph. To achieve this, we propose an object-difference based graph learner, which learns question-adaptive semantic relations by calculating inter-object difference under the guidance of questions. With the learned relationships, the input image can be represented as an object graph encoded with structural dependencies between objects. In addition, existing graph-based models leverage the pre-extracted object boxes by the object detection model as node features for convenience, but they are suffering from the redundancy problem. To reduce the redundant objects, we introduce a soft-attention mechanism to magnify the question-related objects. Moreover, we incorporate our object-difference based graph learner into the soft-attention based Graph Convolutional Networks to capture question-specific objects and their interactions for answer prediction. Our experimental results on the VQA 2.0 dataset demonstrate that our model gives significantly better performance than baseline methods. |
| 资助项目 | National Key Research and Development Program of China[2016QY03D0505] ; National Natural Science Foundation of China[U19A2057] |
| WOS研究方向 | Computer Science ; Engineering |
| 语种 | 英语 |
| WOS记录号 | WOS:000521018700001 |
| 出版者 | SPRINGER |
| 资助机构 | National Key Research and Development Program of China ; National Natural Science Foundation of China |
| 源URL | [http://ir.ia.ac.cn/handle/173211/38696]  |
| 专题 | 数字内容技术与服务研究中心_远程智能医疗 |
| 通讯作者 | Mao, Zhendong |
| 作者单位 | 1.Univ Sci & Technol China, Hefei, Peoples R China 2.China Acad Elect & Informat Technol, Beijing, Peoples R China 3.Xiaomi Inc, Xiaomi AI Lab, Beijing, Peoples R China 4.Chinese Acad Sci, Inst Informat Engn, Beijing, Peoples R China 5.Univ Chinese Acad Sci, Sch Cyber Secur, Beijing, Peoples R China 6.Chinese Acad Sci, Inst Automat, Beijing, Peoples R China |
| 推荐引用方式 GB/T 7714 | Zhu, Xi,Mao, Zhendong,Chen, Zhineng,et al. Object-difference drived graph convolutional networks for visual question answering[J]. MULTIMEDIA TOOLS AND APPLICATIONS,2020:19. |
| APA | Zhu, Xi,Mao, Zhendong,Chen, Zhineng,Li, Yangyang,Wang, Zhaohui,&Wang, Bin.(2020).Object-difference drived graph convolutional networks for visual question answering.MULTIMEDIA TOOLS AND APPLICATIONS,19. |
| MLA | Zhu, Xi,et al."Object-difference drived graph convolutional networks for visual question answering".MULTIMEDIA TOOLS AND APPLICATIONS (2020):19. |
入库方式： OAI收割
来源：自动化研究所
浏览0
下载0
收藏0
其他版本
除非特别说明，本系统中所有内容都受版权保护，并保留所有权利。