中国科学院机构知识库网格
Chinese Academy of Sciences Institutional Repositories Grid
首页 机构 成果 学者
  
  1. CAS IR Grid
  2. 计算技术研究所
  3. 中国科学院计算技术研究所
  4. 中国科学院计算技术研究所期刊论文
  5. 英文
Exp-VQA: Fine-grained facial expression analysis via visual question answering

文献类型：期刊论文

作者Yuan, Yujian1,2; Zeng, Jiabei1,2; Shan, Shiguang1,2
刊名PATTERN RECOGNITION
出版日期2025-12-01
卷号168页码:13
关键词Fine-grained facial expression analysis Emotion classification Facial action unit detection Visual question answering
ISSN号0031-3203
DOI10.1016/j.patcog.2025.111783
英文摘要This paper presents a novel task, facial expression visual question answering (VQA), for fine-grained facial expression analysis across multiple scales. Facial expression VQA interprets facial expressions in a more detailed and comprehensive way than traditional emotion categories or facial action units (AUs). To develop the facial expression VQA model, we finetuned an InstructBLIP using synthesized VQA pairs, forming the Exp-VQA model. These VQA pairs are synthesized leveraging the powerful descriptive ability of GPT-3.5 and a rule-based generator, based on existing annotations on both emotion classification and AU detection datasets. Exp-VQA can describe the facial status of the whole face as well as infer the indicated emotion, detail facial actions in specific regions, and detect individual AU occurrences. Experiments demonstrate the effectiveness of Exp-VQA in describing multi-scale facial expressions, as well as state-of-the-art zero-shot ability in detecting unseen AUs. Furthermore, the training of Exp-VQA enhances its intermediate visual features' performance on both AU detection and emotion classification tasks. The code and trained models are available at: https://github.com/Yujianyuan/Exp-VQA.
资助项目National Natural Science Foundation of China[62176248] ; National Natural Science Foundation of China[U2336213]
WOS研究方向Computer Science ; Engineering
语种英语
WOS记录号WOS:001502075500001
出版者ELSEVIER SCI LTD
源URL[http://119.78.100.204/handle/2XEOYT63/42313]  
专题中国科学院计算技术研究所期刊论文_英文
通讯作者Zeng, Jiabei
作者单位1.Chinese Acad Sci, Inst Comp Technol, Beijing 100190, Peoples R China
2.Univ Chinese Acad Sci, Beijing 100049, Peoples R China
推荐引用方式
GB/T 7714		 Yuan, Yujian,Zeng, Jiabei,Shan, Shiguang. Exp-VQA: Fine-grained facial expression analysis via visual question answering[J]. PATTERN RECOGNITION,2025,168:13.
APA Yuan, Yujian,Zeng, Jiabei,&Shan, Shiguang.(2025).Exp-VQA: Fine-grained facial expression analysis via visual question answering.PATTERN RECOGNITION,168,13.
MLA Yuan, Yujian,et al."Exp-VQA: Fine-grained facial expression analysis via visual question answering".PATTERN RECOGNITION 168(2025):13.

入库方式： OAI收割

来源：计算技术研究所

浏览0
下载0
收藏0
其他版本

除非特别说明，本系统中所有内容都受版权保护，并保留所有权利。