Multimodal Affective States Recognition Based on Multiscale CNNs and Biologically Inspired Decision Fusion Model
文献类型：期刊论文
| 作者 | Zhao, Yuxuan1,2 ; Cao, Xinyan1; Lin, Jinlong1; Yu, Dunshan1; Cao, Xixin1
|
| 刊名 | IEEE TRANSACTIONS ON AFFECTIVE COMPUTING
 |
| 出版日期 | 2023-04-01 |
| 卷号 | 14期号:2页码:1391-1403 |
| ISSN号 | 1949-3045 |
| 关键词 | Physiology Brain modeling Feature extraction Electroencephalography Biological system modeling Convolution Reliability Multimodal affective states recognition convolutional neural network decision fusion model physiological signals |
| DOI | 10.1109/TAFFC.2021.3093923 |
| 通讯作者 | Zhao, Yuxuan(yuxuan.zhao@ia.ac.cn) |
| 英文摘要 | There has been an encouraging progress in the affective states recognition models based on the single-modality signals as electroencephalogram (EEG) signals or peripheral physiological signals in recent years. However, multimodal physiological signals-based affective states recognition methods have not been thoroughly exploited yet. Here we propose Multiscale Convolutional Neural Networks (Multiscale CNNs) and a biologically inspired decision fusion model for multimodal affective states recognition. First, the raw signals are pre-processed with baseline signals. Then, the High Scale CNN and Low Scale CNN in Multiscale CNNs are utilized to predict the probability of affective states output for EEG and each peripheral physiological signal respectively. Finally, the fusion model calculates the reliability of each single-modality signals by the euclidean distance between various class labels and the classification probability from Multiscale CNNs, and the decision is made by the more reliable modality information while other modalities information is retained. We use this model to classify four affective states from the arousal valence plane in the DEAP and AMIGOS dataset. The results show that the fusion model improves the accuracy of affective states recognition significantly compared with the result on single-modality signals, and the recognition accuracy of the fusion result achieve 98.52 and 99.89 percent in the DEAP and AMIGOS dataset respectively. |
| WOS关键词 | EMOTION RECOGNITION ; PHYSIOLOGICAL SIGNALS ; CUE INTEGRATION ; CLASSIFICATION ; INFORMATION ; TEXTURE ; BRAIN |
| WOS研究方向 | Computer Science |
| 语种 | 英语 |
| 出版者 | IEEE-INST ELECTRICAL ELECTRONICS ENGINEERS INC |
| WOS记录号 | WOS:001000299100040 |
| 源URL | [http://ir.ia.ac.cn/handle/173211/53477]  |
| 专题 | 多模态人工智能系统全国重点实验室 |
| 通讯作者 | Zhao, Yuxuan |
| 作者单位 | 1.Peking Univ, Sch Software & Microelect, Beijing 100871, Peoples R China 2.Chinese Acad Sci, Inst Automat, Beijing 100190, Peoples R China |
| 推荐引用方式 GB/T 7714 | Zhao, Yuxuan,Cao, Xinyan,Lin, Jinlong,et al. Multimodal Affective States Recognition Based on Multiscale CNNs and Biologically Inspired Decision Fusion Model[J]. IEEE TRANSACTIONS ON AFFECTIVE COMPUTING,2023,14(2):1391-1403. |
| APA | Zhao, Yuxuan,Cao, Xinyan,Lin, Jinlong,Yu, Dunshan,&Cao, Xixin.(2023).Multimodal Affective States Recognition Based on Multiscale CNNs and Biologically Inspired Decision Fusion Model.IEEE TRANSACTIONS ON AFFECTIVE COMPUTING,14(2),1391-1403. |
| MLA | Zhao, Yuxuan,et al."Multimodal Affective States Recognition Based on Multiscale CNNs and Biologically Inspired Decision Fusion Model".IEEE TRANSACTIONS ON AFFECTIVE COMPUTING 14.2(2023):1391-1403. |
入库方式： OAI收割
来源：自动化研究所
浏览0
下载0
收藏0
其他版本
除非特别说明，本系统中所有内容都受版权保护，并保留所有权利。