中国科学院机构知识库网格
Chinese Academy of Sciences Institutional Repositories Grid
首页 机构 成果 学者
  
  1. CAS IR Grid
  2. 合肥物质科学研究院
  3. 中国科学院合肥物质科学研究院
A convolution-transformer dual branch network for head-pose and occlusion facial expression recognition

文献类型：期刊论文

作者Liang, Xingcan1,2; Xu, Linsen5; Zhang, Wenxiang3; Zhang, Yan4; Liu, Jinfu1,2; Liu, Zhipeng1,2
刊名VISUAL COMPUTER
出版日期2022-02-13
ISSN号0178-2789
关键词Facial expression recognition CNNs Transformers Feature fusion Robust on occlusions and head-pose variations
DOI10.1007/s00371-022-02413-5
通讯作者Xu, Linsen(lsxu@hhu.edu.cn)
英文摘要Facial expression recognition (FER) has attracted much more attention due to its broad range of applications. Occlusions and head-pose variations are two major obstacles for automatic FER. In this paper, we propose a convolution-transformer dual branch network (CT-DBN) that takes advantage of local and global facial information to tackle the real-word occlusions and head-pose variant robust FER. The CT-DBN contains two branches. Taking into account local modeling ability of CNN, the first branch utilizes CNN to capture local edge information. Inspired by transformers' successful application in natural language processing, we employ transformer to the second branch to be responsible for obtaining better global representation. Then, a local-global feature fusion module is proposed to adaptively integrate both features to hybrid features and model the relationship between them. With the help of feature fusion module, our network not only integrates local and global features in an adaptive weighting manner but can also learn the corresponding distinguishable features autonomously. Experimental results under inner-database and cross-database evaluation on four leading facial expression databases illustrate that our proposed CT-DBN outperforms other state-of-the-art methods and achieves robust performance under in-the-wild condition.
资助项目National Key R&D Program of China[2017YFB1303200] ; Jiangsu Special Project for Frontier Leading Base Technology[BK20192004] ; Key Support Project of Dean Fund of Hefei Institutes of Physical Science, CAS[YZJJZX202017] ; Strategic High-tech Innovation Fund of Chinese Academy of Sciences[GQRC-19-15]
WOS研究方向Computer Science
语种英语
出版者SPRINGER
WOS记录号WOS:000754455000001
资助机构National Key R&D Program of China ; Jiangsu Special Project for Frontier Leading Base Technology ; Key Support Project of Dean Fund of Hefei Institutes of Physical Science, CAS ; Strategic High-tech Innovation Fund of Chinese Academy of Sciences
源URL[http://ir.hfcas.ac.cn:8080/handle/334002/127808]  
专题中国科学院合肥物质科学研究院
通讯作者Xu, Linsen
作者单位1.Chinese Acad Sci, Inst Intelligent Machines, Hefei Inst Phys Sci, Hefei 230031, Peoples R China
2.Univ Sci & Technol China, Hefei 230026, Peoples R China
3.Changzhou Univ, Sch Microelect & Control Engn, Changzhou 213164, Peoples R China
4.Anhui Jianzhu Univ, Sch Elect & Informat Engn, Hefei 230009, Peoples R China
5.Hohai Univ, Coll Mech & Elect Engn, Changzhou 213022, Peoples R China
推荐引用方式
GB/T 7714		 Liang, Xingcan,Xu, Linsen,Zhang, Wenxiang,et al. A convolution-transformer dual branch network for head-pose and occlusion facial expression recognition[J]. VISUAL COMPUTER,2022.
APA Liang, Xingcan,Xu, Linsen,Zhang, Wenxiang,Zhang, Yan,Liu, Jinfu,&Liu, Zhipeng.(2022).A convolution-transformer dual branch network for head-pose and occlusion facial expression recognition.VISUAL COMPUTER.
MLA Liang, Xingcan,et al."A convolution-transformer dual branch network for head-pose and occlusion facial expression recognition".VISUAL COMPUTER (2022).

入库方式： OAI收割

来源：合肥物质科学研究院

浏览0
下载0
收藏0
其他版本

除非特别说明，本系统中所有内容都受版权保护，并保留所有权利。