中国科学院机构知识库网格
Chinese Academy of Sciences Institutional Repositories Grid
MESNet: A Convolutional Neural Network for Spotting Multi-Scale Micro-Expression Intervals in Long Videos

文献类型:期刊论文

作者Wang, Su-Jing1,2; He, Ying1; Li, Jingting1; Fu, Xiaolan1,2
刊名IEEE transactions on image processing : a publication of the IEEE Signal Processing Society
出版日期2021
卷号30页码:3956-3969
关键词Convolutional neural network deep learning detection long videos micro-expression spotting
ISSN号1941-0042
DOI10.1109/TIP.2021.3064258
通讯作者Wang, Su-Jing(wangsujing@psych.ac.cn)
英文摘要

Micro-expression spotting is a fundamental step in the micro-expression analysis. This paper proposes a novel network based convolutional neural network (CNN) for spotting multi-scale spontaneous micro-expression intervals in long videos. We named the network as Micro-Expression Spotting Network (MESNet). It is composed of three modules. The first module is a 2+1D Spatiotemporal Convolutional Network, which uses 2D convolution to extract spatial features and 1D convolution to extract temporal features. The second module is a Clip Proposal Network, which gives some proposed micro-expression clips. The last module is a Classification Regression Network, which classifies the proposed clips to micro-expression or not, and further regresses their temporal boundaries. We also propose a novel evaluation metric for spotting micro-expression. Extensive experiments have been conducted on the two long video datasets: CAS(ME)2 and SAMM, and the leave-one-subject-out cross-validation is used to evaluate the spotting performance. Results show that the proposed MESNet effectively enhances the F1-score metric. And comparative results show the proposed MESNet has achieved a good performance, which outperforms other state-of-the-art methods, especially in the SAMM dataset.

资助项目National Natural Science Foundation of China[U19B2032] ; National Natural Science Foundation of China[61772511] ; National Natural Science Foundation of China[62061136001] ; China Postdoctoral Science Foundation[2020M680738] ; National Key Research and Development Project[2018AAA0100205]
WOS研究方向Computer Science ; Engineering
语种英语
WOS记录号WOS:000637528700001
出版者IEEE-INST ELECTRICAL ELECTRONICS ENGINEERS INC
源URL[http://ir.psych.ac.cn/handle/311026/38736]  
专题心理研究所_中国科学院行为科学重点实验室
作者单位1.Key Laboratory of Behavior Sciences, Institute of Psychology, Chinese Academy of Sciences, Beijing, China
2.Department of Psychology, University of Chinese Academy of Sciences, Beijing, China
推荐引用方式
GB/T 7714
Wang, Su-Jing,He, Ying,Li, Jingting,et al. MESNet: A Convolutional Neural Network for Spotting Multi-Scale Micro-Expression Intervals in Long Videos[J]. IEEE transactions on image processing : a publication of the IEEE Signal Processing Society,2021,30:3956-3969.
APA Wang, Su-Jing,He, Ying,Li, Jingting,&Fu, Xiaolan.(2021).MESNet: A Convolutional Neural Network for Spotting Multi-Scale Micro-Expression Intervals in Long Videos.IEEE transactions on image processing : a publication of the IEEE Signal Processing Society,30,3956-3969.
MLA Wang, Su-Jing,et al."MESNet: A Convolutional Neural Network for Spotting Multi-Scale Micro-Expression Intervals in Long Videos".IEEE transactions on image processing : a publication of the IEEE Signal Processing Society 30(2021):3956-3969.

入库方式: OAI收割

来源:心理研究所

浏览0
下载0
收藏0
其他版本

除非特别说明,本系统中所有内容都受版权保护,并保留所有权利。