Micro-expressions are facial movements that are extremely short and not easily perceived, revealing the individual's hidden real emotions, and could be widely used in lies detection and other fields. The automatic research of micro-expression spotting is mainly limited by the small sample size. This project will address this problem by comprehensively using computer vision technology and cognitive psychology experimental methods. First, a behavioral-experimental paradigm combining eye-movement techniques and a presentation-judgment paradigm with subthreshold emotion priming effects was used to examine the cognitive mechanisms of selective attention allocation in micro-expression recognition and to refine the characteristic regions of interest in human recognition of micro-expressions. Second, based on the human attention mechanism, we propose a micro-expression spotting method based on a multi-branching self-supervised learning network, extracting structure-based, detail, spatio-temporal variation, and depth features of video samples. This research will achieve theoretical and technological breakthroughs in the field of automatic micro-expression spotting, and lay the foundation for the application of micro-expression spotting in realistic and complex scenarios.

微表情是一种持续时间极短、不易被察觉的面部动作,揭示了个体的真实情绪,可以被广泛地应用于谎言识别等领域。而微表情检测的研究受到小样本问题的限制。针对该问题,本文结合计算机视觉技术与认知心理学实验方法进行探索。首先,结合眼动技术和呈现-判断范式与阈下情绪启动效应的行为实验范式,考察微表情识别中选择注意分配的认知机制,细化人类识别微表情时的特征兴趣区域。其次,结合人类注意机制,提出基于自监督学习的多模态微表情检测方法。通过理论和关键技术的突破,为真实场景下微表情检测的应用奠定基础。

国家自然科学基金项目(62106256、U19B2032);; 中国人民公安大学公共安全行为科学实验室开放课题(2020SYS12);; 中国博士后科学基金项目(2020M680738)资助