MESNet: A Convolutional Neural Network for Spotting Multi-Scale Micro-Expression Intervals in Long Videos
文献类型:期刊论文
作者 | Wang, Su-Jing1,2![]() ![]() |
刊名 | IEEE transactions on image processing : a publication of the IEEE Signal Processing Society
![]() |
出版日期 | 2021 |
卷号 | 30页码:3956-3969 |
关键词 | Convolutional neural network deep learning detection long videos micro-expression spotting |
ISSN号 | 1941-0042 |
DOI | 10.1109/TIP.2021.3064258 |
通讯作者 | Wang, Su-Jing(wangsujing@psych.ac.cn) |
英文摘要 | Micro-expression spotting is a fundamental step in the micro-expression analysis. This paper proposes a novel network based convolutional neural network (CNN) for spotting multi-scale spontaneous micro-expression intervals in long videos. We named the network as Micro-Expression Spotting Network (MESNet). It is composed of three modules. The first module is a 2+1D Spatiotemporal Convolutional Network, which uses 2D convolution to extract spatial features and 1D convolution to extract temporal features. The second module is a Clip Proposal Network, which gives some proposed micro-expression clips. The last module is a Classification Regression Network, which classifies the proposed clips to micro-expression or not, and further regresses their temporal boundaries. We also propose a novel evaluation metric for spotting micro-expression. Extensive experiments have been conducted on the two long video datasets: CAS(ME)2 and SAMM, and the leave-one-subject-out cross-validation is used to evaluate the spotting performance. Results show that the proposed MESNet effectively enhances the F1-score metric. And comparative results show the proposed MESNet has achieved a good performance, which outperforms other state-of-the-art methods, especially in the SAMM dataset. |
资助项目 | National Natural Science Foundation of China[U19B2032] ; National Natural Science Foundation of China[61772511] ; National Natural Science Foundation of China[62061136001] ; China Postdoctoral Science Foundation[2020M680738] ; National Key Research and Development Project[2018AAA0100205] |
WOS研究方向 | Computer Science ; Engineering |
语种 | 英语 |
WOS记录号 | WOS:000637528700001 |
出版者 | IEEE-INST ELECTRICAL ELECTRONICS ENGINEERS INC |
源URL | [http://ir.psych.ac.cn/handle/311026/38736] ![]() |
专题 | 心理研究所_中国科学院行为科学重点实验室 |
作者单位 | 1.Key Laboratory of Behavior Sciences, Institute of Psychology, Chinese Academy of Sciences, Beijing, China 2.Department of Psychology, University of Chinese Academy of Sciences, Beijing, China |
推荐引用方式 GB/T 7714 | Wang, Su-Jing,He, Ying,Li, Jingting,et al. MESNet: A Convolutional Neural Network for Spotting Multi-Scale Micro-Expression Intervals in Long Videos[J]. IEEE transactions on image processing : a publication of the IEEE Signal Processing Society,2021,30:3956-3969. |
APA | Wang, Su-Jing,He, Ying,Li, Jingting,&Fu, Xiaolan.(2021).MESNet: A Convolutional Neural Network for Spotting Multi-Scale Micro-Expression Intervals in Long Videos.IEEE transactions on image processing : a publication of the IEEE Signal Processing Society,30,3956-3969. |
MLA | Wang, Su-Jing,et al."MESNet: A Convolutional Neural Network for Spotting Multi-Scale Micro-Expression Intervals in Long Videos".IEEE transactions on image processing : a publication of the IEEE Signal Processing Society 30(2021):3956-3969. |
入库方式: OAI收割
来源:心理研究所
其他版本
除非特别说明,本系统中所有内容都受版权保护,并保留所有权利。