Realistic Facial Expression Reconstruction for VR HMD Users
文献类型：期刊论文
| 作者 | Lou, Jianwen2; Wang, Yiming2; Nduka, Charles3; Hamedi, Mahyar3; Mavridou, Ifigeneia3; Wang, Fei-Yue1 ; Yu, Hui2
|
| 刊名 | IEEE TRANSACTIONS ON MULTIMEDIA
 |
| 出版日期 | 2020-03-01 |
| 卷号 | 22期号:3页码:730-743 |
| 关键词 | Face Sensors Three-dimensional displays Image reconstruction Resists Electromyography Cameras Facial expression reconstruction head-mounted display electromyogram 3D face reconstruction facial action unit |
| ISSN号 | 1520-9210 |
| DOI | 10.1109/TMM.2019.2933338 |
| 通讯作者 | Yu, Hui(hui.yu@port.ac.uk) |
| 英文摘要 | We present a system for sensing and reconstructing facial expressions of the virtual reality (VR) head-mounted display (HMD) user. The HMD occludes a large portion of the user's face, which makes most existing facial performance capturing techniques intractable. To tackle this problem, a novel hardware solution with electromyography (EMG) sensors being attached to the headset frame is applied to track facial muscle movements. For realistic facial expression recovery, we first reconstruct the user's 3D face from a single image and generate the personalized blendshapes associated with seven facial action units (AUs) on the most emotionally salient facial parts (ESFPs). We then utilize pre-processed EMG signals for measuring activations of AU-coded facial expressions to drive pre-built personalized blendshapes. Since facial expressions appear as important nonverbal cues of the subject's internal emotional states, we further investigate the relationship between six basic emotions - anger, disgust, fear, happiness, sadness and surprise, and detected AUs using a fern classifier. Experiments show the proposed system can accurately sense and reconstruct high-fidelity common facial expressions while providing useful information regarding the emotional state of the HMD user. |
| WOS关键词 | 3D FACE RECONSTRUCTION ; RECOGNITION ; PERCEPTION ; FRAMEWORK ; DATABASE ; MACHINE |
| 资助项目 | Engineering and Physical Sciences Research Council[EP/N025849/1] ; Royal Academy of Engineering Grant[IFS1819\9] ; Emteq Ltd. |
| WOS研究方向 | Computer Science ; Telecommunications |
| 语种 | 英语 |
| WOS记录号 | WOS:000519576700013 |
| 出版者 | IEEE-INST ELECTRICAL ELECTRONICS ENGINEERS INC |
| 资助机构 | Engineering and Physical Sciences Research Council ; Royal Academy of Engineering Grant ; Emteq Ltd. |
| 源URL | [http://ir.ia.ac.cn/handle/173211/38594]  |
| 专题 | 自动化研究所_复杂系统管理与控制国家重点实验室_先进控制与自动化团队 |
| 通讯作者 | Yu, Hui |
| 作者单位 | 1.Chinese Acad Sci, Inst Automat, State Key Lab Management & Control Complex Syst, Beijing 100190, Peoples R China 2.Univ Portsmouth, Sch Creat Technol, Portsmouth PO1 2DJ, Hants, England 3.Emteq Ltd, Sussex Innovat Ctr, Brighton BN1 9SB, E Sussex, England |
| 推荐引用方式 GB/T 7714 | Lou, Jianwen,Wang, Yiming,Nduka, Charles,et al. Realistic Facial Expression Reconstruction for VR HMD Users[J]. IEEE TRANSACTIONS ON MULTIMEDIA,2020,22(3):730-743. |
| APA | Lou, Jianwen.,Wang, Yiming.,Nduka, Charles.,Hamedi, Mahyar.,Mavridou, Ifigeneia.,...&Yu, Hui.(2020).Realistic Facial Expression Reconstruction for VR HMD Users.IEEE TRANSACTIONS ON MULTIMEDIA,22(3),730-743. |
| MLA | Lou, Jianwen,et al."Realistic Facial Expression Reconstruction for VR HMD Users".IEEE TRANSACTIONS ON MULTIMEDIA 22.3(2020):730-743. |
入库方式： OAI收割
来源：自动化研究所
浏览0
下载0
收藏0
其他版本
除非特别说明，本系统中所有内容都受版权保护，并保留所有权利。