中国科学院机构知识库网格
Chinese Academy of Sciences Institutional Repositories Grid
首页 机构 成果 学者
  
  1. CAS IR Grid
  2. 计算技术研究所
  3. 中国科学院计算技术研究所
  4. 中国科学院计算技术研究所期刊论文
  5. 英文
Contrastive Learning of Person-Independent Representations for Facial Action Unit Detection

文献类型：期刊论文

作者Li, Yong2; Shan, Shiguang1,3,4
刊名IEEE TRANSACTIONS ON IMAGE PROCESSING
出版日期2023
卷号32页码:3212-3225
ISSN号1057-7149
关键词Gold Videos Training Image reconstruction Feature extraction Faces Task analysis Facial action unit detection contrastive Learning self-supervised learning person-independent action unit detection
DOI10.1109/TIP.2023.3279978
英文摘要Facial action unit (AU) detection, aiming to classify AU present in the facial image, has long suffered from insufficient AU annotations. In this paper, we aim to mitigate this data scarcity issue by learning AU representations from a large number of unlabelled facial videos in a contrastive learning paradigm. We formulate the self-supervised AU representation learning signals in two-fold: 1) AU representation should be frame-wisely discriminative within a short video clip; 2) Facial frames sampled from different identities but show analogous facial AUs should have consistent AU representations. As to achieve these goals, we propose to contrastively learn the AU representation within a video clip and devise a cross-identity reconstruction mechanism to learn the person-independent representations. Specially, we adopt a margin-based temporal contrastive learning paradigm to perceive the temporal AU coherence and evolution characteristics within a clip that consists of consecutive input facial frames. Moreover, the cross-identity reconstruction mechanism facilitates pushing the faces from different identities but show analogous AUs close in the latent embedding space. Experimental results on three public AU datasets demonstrate that the learned AU representation is discriminative for AU detection. Our method outperforms other contrastive learning methods and significantly closes the performance gap between the self-supervised and supervised AU detection approaches.
资助项目National Key Research and Development Program of China[2018AAA0102402] ; National Natural Science Foundation of China[62102180] ; Natural Science Foundation of Jiangsu Province[BK20210329] ; Shuangchuang Program of Jiangsu Province[JSSCBS20210210]
WOS研究方向Computer Science ; Engineering
语种英语
出版者IEEE-INST ELECTRICAL ELECTRONICS ENGINEERS INC
WOS记录号WOS:001004183400002
源URL[http://119.78.100.204/handle/2XEOYT63/21213]  
专题中国科学院计算技术研究所期刊论文_英文
通讯作者Shan, Shiguang
作者单位1.Peng Cheng Lab, Shenzhen 518055, Peoples R China
2.Nanjing Univ Sci & Technol, Sch Comp Sci & Engn, Key Lab Intelligent Percept & Syst High Dimens Inf, Minist Educ, Nanjing 210094, Peoples R China
3.Chinese Acad Sci, Inst Comp Technol, Key Lab Intelligent Informat Proc, Beijing 100190, Peoples R China
4.Univ Chinese Acad Sci, Sch Comp Sci & Technol, Beijing 100049, Peoples R China
推荐引用方式
GB/T 7714		 Li, Yong,Shan, Shiguang. Contrastive Learning of Person-Independent Representations for Facial Action Unit Detection[J]. IEEE TRANSACTIONS ON IMAGE PROCESSING,2023,32:3212-3225.
APA Li, Yong,&Shan, Shiguang.(2023).Contrastive Learning of Person-Independent Representations for Facial Action Unit Detection.IEEE TRANSACTIONS ON IMAGE PROCESSING,32,3212-3225.
MLA Li, Yong,et al."Contrastive Learning of Person-Independent Representations for Facial Action Unit Detection".IEEE TRANSACTIONS ON IMAGE PROCESSING 32(2023):3212-3225.

入库方式： OAI收割

来源：计算技术研究所

浏览0
下载0
收藏0
其他版本

除非特别说明，本系统中所有内容都受版权保护，并保留所有权利。