A multimodal approach to estimating vigilance in SSVEP-based BCI
文献类型：期刊论文
| 作者 | Wang, Kangning2,3; Qiu, Shuang3,4 ; Wei, Wei3; Zhang, Yukun3,4; Wang, Shengpei3; He, Huiguang4; Xu, Minpeng2,5; Jung, Tzyy-Ping1,2,5; Ming, Dong2,5
|
| 刊名 | EXPERT SYSTEMS WITH APPLICATIONS
 |
| 出版日期 | 2023-09-01 |
| 卷号 | 225页码:16 |
| ISSN号 | 0957-4174 |
| 关键词 | Vigilance estimation Brain -computer interface (BCI) Graph neural network Electroencephalogram (EEG) Steady-state visual evoked potential (SSVEP) Multimodal fusion |
| DOI | 10.1016/j.eswa.2023.120177 |
| 通讯作者 | Qiu, Shuang(shuang.qiu@ia.ac.cn) ; Ming, Dong(richardming@tju.edu.cn) |
| 英文摘要 | Brain-computer interface (BCI) is a communication system that allows a direct connection between the human brain and external devices, which is able to provide assistance and improve the quality of life for people with disabilities. Vigilance is an important cognitive state and plays an important role in human-computer interac-tion. In BCI tasks, the low-vigilance state of the BCI user would lead to the performance degradation. Therefore, it is desirable to develop an efficient method to estimate the vigilance state of BCI users. In this study, we built a 4 -target BCI system based on steady-state visual evoked potential (SSVEP) for cursor control. Electroencephalo-gram (EEG) and electrooculogram (EOG) were recorded simultaneously from 18 subjects during a 90-min continuous cursor-control BCI task. We proposed a multimodal vigilance estimating network, named MVENet, to estimate the vigilance state of BCI users through the multimodal signals. In this architecture, a spatial -temporal convolution module with an attention mechanism was adopted to explore the temporal-spatial infor-mation of the EEG features, and a long short-term memory module was utilized to learn the temporal de-pendencies of EOG features. Moreover, a fusion mechanism was built to fuse the EEG representations and EOG representations effectively. Experimental results showed that the proposed network achieved a better perfor-mance than the compared methods. These results demonstrate the feasibility and effectiveness of our methods for estimating the vigilance state of BCI users. |
| WOS关键词 | EEG ; ATTENTION ; DECREMENT ; SYSTEM ; LEVEL |
| 资助项目 | Beijing Natural Science Foundation[7222311] ; Beijing Natural Science Foundation[J210010] ; National Natural Sci- ence Foundation of China[U21A20388] ; National Natural Sci- ence Foundation of China[62276262] ; National Natural Sci- ence Foundation of China[62206285] ; National Natural Sci- ence Foundation of China[62201569] |
| WOS研究方向 | Computer Science ; Engineering ; Operations Research & Management Science |
| 语种 | 英语 |
| 出版者 | PERGAMON-ELSEVIER SCIENCE LTD |
| WOS记录号 | WOS:000989204800001 |
| 资助机构 | Beijing Natural Science Foundation ; National Natural Sci- ence Foundation of China |
| 源URL | [http://ir.ia.ac.cn/handle/173211/53377]  |
| 专题 | 多模态人工智能系统全国重点实验室 |
| 通讯作者 | Qiu, Shuang; Ming, Dong |
| 作者单位 | 1.Univ Calif, Swartz Ctr Computat Neurosci, San Diego, CA 92093 USA 2.Tianjin Univ, Acad Med Engn & Translat Med, Tianjin 300072, Peoples R China 3.Chinese Acad Sci, Inst Automat, Lab Brain Atlas & Brain Inspired Intelligence, State Key Lab Multimodal Artificial Intelligence S, Beijing 100190, Peoples R China 4.Univ Chinese Acad Sci, Sch Artificial Intelligence, Beijing 100049, Peoples R China 5.Tianjin Univ, Coll Precis Instruments & Optoelect Engn, Tianjin 300072, Peoples R China |
| 推荐引用方式 GB/T 7714 | Wang, Kangning,Qiu, Shuang,Wei, Wei,et al. A multimodal approach to estimating vigilance in SSVEP-based BCI[J]. EXPERT SYSTEMS WITH APPLICATIONS,2023,225:16. |
| APA | Wang, Kangning.,Qiu, Shuang.,Wei, Wei.,Zhang, Yukun.,Wang, Shengpei.,...&Ming, Dong.(2023).A multimodal approach to estimating vigilance in SSVEP-based BCI.EXPERT SYSTEMS WITH APPLICATIONS,225,16. |
| MLA | Wang, Kangning,et al."A multimodal approach to estimating vigilance in SSVEP-based BCI".EXPERT SYSTEMS WITH APPLICATIONS 225(2023):16. |
入库方式： OAI收割
来源：自动化研究所
浏览0
下载0
收藏0
其他版本
除非特别说明，本系统中所有内容都受版权保护，并保留所有权利。