中国科学院机构知识库网格
Chinese Academy of Sciences Institutional Repositories Grid
首页 机构 成果 学者
  
  1. CAS IR Grid
  2. 计算技术研究所
  3. 中国科学院计算技术研究所
  4. 中国科学院计算技术研究所期刊论文
  5. 英文
CRIC: A VQA Dataset for Compositional Reasoning on Vision and Commonsense

文献类型：期刊论文

作者Gao, Difei; Wang, Ruiping; Shan, Shiguang; Chen, Xilin
刊名IEEE TRANSACTIONS ON PATTERN ANALYSIS AND MACHINE INTELLIGENCE
出版日期2023-05-01
卷号45期号:5页码:5561-5578
ISSN号0162-8828
关键词Visualization Task analysis Tail Head Annotations Magnetic heads Mouth Visual question answering compositional reasoning commonsense reasoning dataset construction
DOI10.1109/TPAMI.2022.3210780
英文摘要Alternatively inferring on the visual facts and commonsense is fundamental for an advanced visual question answering (VQA) system. This ability requires models to go beyond the literal understanding of commonsense. The system should not just treat objects as the entrance to query background knowledge, but fully ground commonsense to the visual world and imagine the possible relationships between objects, e.g., "fork, can lift, food". To comprehensively evaluate such abilities, we propose a VQA benchmark, Compositional Reasoning on vIsion and Commonsense(CRIC), which introduces new types of questions about CRIC, and an evaluation metric integrating the correctness of answering and commonsense grounding. To collect such questions and rich additional annotations to support the metric, we also propose an automatic algorithm to generate question samples from the scene graph associated with the images and the relevant knowledge graph. We further analyze several representative types of VQA models on the CRIC dataset. Experimental results show that grounding the commonsense to the image region and joint reasoning on vision and commonsense are still challenging for current approaches. The dataset is available at https://cricvqa.github.io.
资助项目National Key R&D Program of China[2021ZD0111901] ; Natural Science Foundation of China[U21B2025] ; Natural Science Foundation of China[U19B2036] ; Natural Science Foundation of China[61922080]
WOS研究方向Computer Science ; Engineering
语种英语
出版者IEEE COMPUTER SOC
WOS记录号WOS:000964792800015
源URL[http://119.78.100.204/handle/2XEOYT63/21407]  
专题中国科学院计算技术研究所期刊论文_英文
通讯作者Wang, Ruiping
作者单位Chinese Acad Sci, Inst Comp Technol, Key Lab Intelligent Informat Proc, Beijing 100190, Peoples R China
推荐引用方式
GB/T 7714		 Gao, Difei,Wang, Ruiping,Shan, Shiguang,et al. CRIC: A VQA Dataset for Compositional Reasoning on Vision and Commonsense[J]. IEEE TRANSACTIONS ON PATTERN ANALYSIS AND MACHINE INTELLIGENCE,2023,45(5):5561-5578.
APA Gao, Difei,Wang, Ruiping,Shan, Shiguang,&Chen, Xilin.(2023).CRIC: A VQA Dataset for Compositional Reasoning on Vision and Commonsense.IEEE TRANSACTIONS ON PATTERN ANALYSIS AND MACHINE INTELLIGENCE,45(5),5561-5578.
MLA Gao, Difei,et al."CRIC: A VQA Dataset for Compositional Reasoning on Vision and Commonsense".IEEE TRANSACTIONS ON PATTERN ANALYSIS AND MACHINE INTELLIGENCE 45.5(2023):5561-5578.

入库方式： OAI收割

来源：计算技术研究所

浏览0
下载0
收藏0
其他版本

除非特别说明，本系统中所有内容都受版权保护，并保留所有权利。