Coordinating explicit and implicit knowledge for knowledge-based VQA
文献类型：期刊论文
| 作者 | Wang, Qunbo1; Liu, Jing1 ; Wu, Wenjun2
|
| 刊名 | PATTERN RECOGNITION
 |
| 出版日期 | 2024-07-01 |
| 卷号 | 151页码:9 |
| 关键词 | Knowledge retrieval Pre -trained model Knowledge -based VQA |
| ISSN号 | 0031-3203 |
| DOI | 10.1016/j.patcog.2024.110368 |
| 通讯作者 | Liu, Jing(jliu@nlpr.ia.ac.cn) |
| 英文摘要 | Pre -trained models often generate plausible looking statements that are factually incorrect because of the inaccurate implicit knowledge contained in the model's parameters. Related methods retrieve explicit knowledge from the external knowledge source to help improve the prediction performance and reliability. However, these methods often use weak training signals for the retriever, and require the model to make each prediction based on the retrieved knowledge, even when the retrieved knowledge is not reliable or the model can produce better prediction only using its implicit knowledge. Therefore, it is necessary to enable the pre -trained model to actively select more beneficial knowledge for producing better prediction. This work proposes a novel method to help the model to Coordinate Explicit and Implicit Knowledge (CEIK) for the knowledge -based visual question answering (VQA) task, which is an important direction of pre -trained models. Furthermore, a better training signal is proposed for the retriever according to whether the retrieved knowledge can correct the prediction. Experimental results demonstrate the effectiveness of our method. |
| 资助项目 | National Key R&D Program of China[2022ZD0118802] ; National Natural Science Foundation of China[62206279] |
| WOS研究方向 | Computer Science ; Engineering |
| 语种 | 英语 |
| WOS记录号 | WOS:001195942400001 |
| 出版者 | ELSEVIER SCI LTD |
| 资助机构 | National Key R&D Program of China ; National Natural Science Foundation of China |
| 源URL | [http://ir.ia.ac.cn/handle/173211/58044]  |
| 专题 | 自动化研究所_模式识别国家重点实验室_图像与视频分析团队 |
| 通讯作者 | Liu, Jing |
| 作者单位 | 1.Chinese Acad Sci, Inst Automat, Beijing 100191, Peoples R China 2.Beihang Univ, Inst Artificial Intelligence, Beijing 100191, Peoples R China |
| 推荐引用方式 GB/T 7714 | Wang, Qunbo,Liu, Jing,Wu, Wenjun. Coordinating explicit and implicit knowledge for knowledge-based VQA[J]. PATTERN RECOGNITION,2024,151:9. |
| APA | Wang, Qunbo,Liu, Jing,&Wu, Wenjun.(2024).Coordinating explicit and implicit knowledge for knowledge-based VQA.PATTERN RECOGNITION,151,9. |
| MLA | Wang, Qunbo,et al."Coordinating explicit and implicit knowledge for knowledge-based VQA".PATTERN RECOGNITION 151(2024):9. |
入库方式： OAI收割
来源：自动化研究所
浏览0
下载0
收藏0
其他版本
除非特别说明，本系统中所有内容都受版权保护，并保留所有权利。