Integrative interaction of emotional speech in audio-visual modality
文献类型：期刊论文
| 作者 | Dong, Haibin2; Li, Na2 ; Fan, Lingzhong1; Wei, Jianguo2; Xu, Junhai2
|
| 刊名 | FRONTIERS IN NEUROSCIENCE
 |
| 出版日期 | 2022-11-11 |
| 卷号 | 16页码:13 |
| 关键词 | audio-visual integration emotional speech fMRI left insula weighted RSA |
| DOI | 10.3389/fnins.2022.797277 |
| 通讯作者 | Xu, Junhai(jhxu@tju.edu.cn) |
| 英文摘要 | Emotional clues are always expressed in many ways in our daily life, and the emotional information we receive is often represented by multiple modalities. Successful social interactions require a combination of multisensory cues to accurately determine the emotion of others. The integration mechanism of multimodal emotional information has been widely investigated. Different brain activity measurement methods were used to determine the location of brain regions involved in the audio-visual integration of emotional information, mainly in the bilateral superior temporal regions. However, the methods adopted in these studies are relatively simple, and the materials of the study rarely contain speech information. The integration mechanism of emotional speech in the human brain still needs further examinations. In this paper, a functional magnetic resonance imaging (fMRI) study was conducted using event-related design to explore the audio-visual integration mechanism of emotional speech in the human brain by using dynamic facial expressions and emotional speech to express emotions of different valences. Representational similarity analysis (RSA) based on regions of interest (ROIs), whole brain searchlight analysis, modality conjunction analysis and supra-additive analysis were used to analyze and verify the role of relevant brain regions. Meanwhile, a weighted RSA method was used to evaluate the contributions of each candidate model in the best fitted model of ROIs. The results showed that only the left insula was detected by all methods, suggesting that the left insula played an important role in the audio-visual integration of emotional speech. Whole brain searchlight analysis, modality conjunction analysis and supra-additive analysis together revealed that the bilateral middle temporal gyrus (MTG), right inferior parietal lobule and bilateral precuneus might be involved in the audio-visual integration of emotional speech from other aspects. |
| WOS关键词 | SUPERIOR TEMPORAL SULCUS ; HUMAN BRAIN ; PERCEPTION ; FACE ; INFORMATION ; EXPRESSIONS ; ACTIVATION ; PRECUNEUS ; INSULA ; VOICE |
| 资助项目 | National Natural Science Foundation of China ; China Postdoctoral Science Foundation ; Project of Qinghai Science and Technology Program ; [62176181] ; [2020M680905] ; [2022-ZJ-T05] |
| WOS研究方向 | Neurosciences & Neurology |
| 语种 | 英语 |
| WOS记录号 | WOS:000890344700001 |
| 出版者 | FRONTIERS MEDIA SA |
| 资助机构 | National Natural Science Foundation of China ; China Postdoctoral Science Foundation ; Project of Qinghai Science and Technology Program |
| 源URL | [http://ir.ia.ac.cn/handle/173211/51290]  |
| 专题 | 自动化研究所_脑网络组研究中心 |
| 通讯作者 | Xu, Junhai |
| 作者单位 | 1.Chinese Acad Sci, Inst Automat, Brainnetome Ctr, Beijing, Peoples R China 2.Tianjin Univ, Coll Intelligence & Comp, Tianjin Key Lab Cognit Comp & Applicat, Tianjin, Peoples R China |
| 推荐引用方式 GB/T 7714 | Dong, Haibin,Li, Na,Fan, Lingzhong,et al. Integrative interaction of emotional speech in audio-visual modality[J]. FRONTIERS IN NEUROSCIENCE,2022,16:13. |
| APA | Dong, Haibin,Li, Na,Fan, Lingzhong,Wei, Jianguo,&Xu, Junhai.(2022).Integrative interaction of emotional speech in audio-visual modality.FRONTIERS IN NEUROSCIENCE,16,13. |
| MLA | Dong, Haibin,et al."Integrative interaction of emotional speech in audio-visual modality".FRONTIERS IN NEUROSCIENCE 16(2022):13. |
入库方式： OAI收割
来源：自动化研究所
浏览0
下载0
收藏0
其他版本
除非特别说明，本系统中所有内容都受版权保护，并保留所有权利。