Knowledge-Embedded Mutual Guidance for Visual Reasoning
文献类型：期刊论文
| 作者 | Zheng, Wenbo3,7; Yan, Lan1,2 ; Chen, Long4,5; Li, Qiang6; Wang, Fei-Yue6
|
| 刊名 | IEEE TRANSACTIONS ON CYBERNETICS
 |
| 出版日期 | 2023-09-20 |
| 页码 | 13 |
| 关键词 | Attention model joint learning knowledge embedding visual reasoning |
| ISSN号 | 2168-2267 |
| DOI | 10.1109/TCYB.2023.3310892 |
| 通讯作者 | Zheng, Wenbo(zwb2022@whut.edu.cn) |
| 英文摘要 | Visual reasoning between visual images and natural language is a long-standing challenge in computer vision. Most of the methods aim to look for answers to questions only on the basis of the analysis of the offered questions and images. Other approaches treat knowledge graphs as flattened tables to search for the answer. However, there are two major problems with these works: 1) the model disregards the fact that the world we surrounding us interlinks our hearing and speaking of natural language and 2) the model largely ignores the structure of the KG. To overcome these challenging deficiencies, a model should jointly consider two modalities of vision and language, as well as the rich structural and logical information embedded in knowledge graphs. To this end, we propose a general joint representation learning framework for visual reasoning, namely, knowledge-embedded mutual guidance. It realizes mutual guidance not only between visual data and natural language descriptions but also between knowledge graphs and reasoning models. In addition, it exploits the knowledge derived from the reasoning model to boost knowledge graphs when applying the visual relation detection task. The experimental results demonstrate that the proposed approach performs dramatically better than state-of-the-art methods on two benchmarks for visual reasoning. |
| 资助项目 | Natural Science Foundation of China[62303361] ; Natural Science Foundation of China[62302161] ; Natural Science Foundation of China[U1811463] ; Hainan Provincial Natural Science Foundation of China[623QN266] ; Fundamental Research Funds for the Central Universities[WUT: 233110002] ; National Key Research and Development Program of China[2018AAA0101502] |
| WOS研究方向 | Automation & Control Systems ; Computer Science |
| 语种 | 英语 |
| WOS记录号 | WOS:001071913500001 |
| 出版者 | IEEE-INST ELECTRICAL ELECTRONICS ENGINEERS INC |
| 资助机构 | Natural Science Foundation of China ; Hainan Provincial Natural Science Foundation of China ; Fundamental Research Funds for the Central Universities ; National Key Research and Development Program of China |
| 源URL | [http://ir.ia.ac.cn/handle/173211/53140]  |
| 专题 | 多模态人工智能系统全国重点实验室 |
| 通讯作者 | Zheng, Wenbo |
| 作者单位 | 1.Hunan Univ, Coll Comp Sci & Engn, Changsha 410082, Peoples R China 2.Natl Supercomp Ctr Changsha, Changsha 410082, Hunan, Peoples R China 3.Wuhan Univ Technol, Sch Comp Sci & Artificial Intelligence, Wuhan 430070, Peoples R China 4.Chinese Acad Sci, Inst Automat, State Key Lab Management & Control Complex Syst, Beijing 100864, Peoples R China 5.Waytous Inc, Beijing 100864, Peoples R China 6.Chinese Acad Sci, Inst Automat, State Key Lab Management & Control Complex Syst, Beijing 100190, Peoples R China 7.Wuhan Univ Technol, Sanya Sci & Educ Innovat Pk, Sanya 572000, Peoples R China |
| 推荐引用方式 GB/T 7714 | Zheng, Wenbo,Yan, Lan,Chen, Long,et al. Knowledge-Embedded Mutual Guidance for Visual Reasoning[J]. IEEE TRANSACTIONS ON CYBERNETICS,2023:13. |
| APA | Zheng, Wenbo,Yan, Lan,Chen, Long,Li, Qiang,&Wang, Fei-Yue.(2023).Knowledge-Embedded Mutual Guidance for Visual Reasoning.IEEE TRANSACTIONS ON CYBERNETICS,13. |
| MLA | Zheng, Wenbo,et al."Knowledge-Embedded Mutual Guidance for Visual Reasoning".IEEE TRANSACTIONS ON CYBERNETICS (2023):13. |
入库方式： OAI收割
来源：自动化研究所
浏览0
下载0
收藏0
其他版本
除非特别说明，本系统中所有内容都受版权保护，并保留所有权利。