Learning to Answer Complex Visual Questions from Multi-View Analysis
文献类型:会议论文
作者 | Zhu MJ(朱敏郡); Weng YX(翁诣轩); He SZ(何世柱); Liu K(刘康); Zhao J(赵军) |
出版日期 | 2022-08 |
会议日期 | 2022 |
会议地点 | 中国秦皇岛 |
英文摘要 | Visual Question Answering (VQA) has received increasing attention in NLP research. Most VQA images focus on natural scenes. However, some images widely used in textbooks such as diagrams often contain complicated and abstract information (e.g. constructed graphs with logic and concepts). Therefore, Diagram Question answering (DQA) is a challenging but significant task, which is also helpful for machines to understand human cognitive behaviors and learning habits. On DQA task, we propose a multi-perspective understanding based visual question-answering method, which constructs a variety of different self-monitoring tasks in the form of prompts to help the model learn deeper information. For the first time, we propose a decoding method of "Cross Entropy constraint Decoding", which can effectively constrain the content generated by the text when performing multiple selection tasks. This method has obtained SOTA in the evaluation task of CCKS-2022, which fully proves the effectiveness of the method. |
会议录出版者 | IEEE |
源URL | [http://ir.ia.ac.cn/handle/173211/52286] |
专题 | 模式识别国家重点实验室_自然语言处理 |
通讯作者 | Zhao J(赵军) |
作者单位 | 中国科学院自动化研究所 |
推荐引用方式 GB/T 7714 | Zhu MJ,Weng YX,He SZ,et al. Learning to Answer Complex Visual Questions from Multi-View Analysis[C]. 见:. 中国秦皇岛. 2022. |
入库方式: OAI收割
来源:自动化研究所
其他版本
除非特别说明,本系统中所有内容都受版权保护,并保留所有权利。