Multi-Level Pixel-Wise Correspondence Learning for 6DoF Face Pose Estimation
文献类型：期刊论文
| 作者 | Xu M(徐淼)2 ; Xiangyu Zhu2; Yueying Kao1; Zhiwen Chen1; Jiangjing Lyu1; Zhen Lei2
|
| 刊名 | IEEE Transactions on Multimedia
 |
| 出版日期 | 2024 |
| 页码 | IEEE Xplore |
| 英文摘要 | Abstract—In this paper, we focus on estimating six degrees
of freedom (6DoF) pose of a face from a single RGB image,
which is an important but under-investigated problem in 3D face
applications such as face reconstruction, forgery detection and
virtual try-on. This problem is different from traditional face
pose estimation and 3D face reconstruction since the distance
from camera to face should be estimated, which can not be
directly regressed due to the non-linearity of the pose space.
To solve the problem, we follow Perspective-n-Point (PnP) and
predict the correspondences between 3D points in canonical space
and 2D facial pixels on the input image to solve the 6DoF
pose parameters. In this framework, the central problem of
6DoF estimation is building the correspondence matrix between
a set of sampled 2D pixels and 3D points, and we propose a
Correspondence Learning Transformer (CLT) to achieve this
goal. Specifically, we build the 2D and 3D features with local,
global, and semantic information, and employ self-attention to
make the 2D and 3D features interact with each other and
build the 2D-3D correspondence. Besides, we argue that 6DoF
estimation is not only related with face appearance itself but
also the facial external context, which contains rich information
about the distance to camera. Therefore, we extract global
and-local features from the integration of face and context,
where the cropped face image with smaller receptive fields
concentrates on the small distortion by perspective projection,
and the whole image with large receptive field provides shoulder
and environment information. Experiments show that our method
achieves a 2.0% improvement of MAEr and ADD on ARKitFace
and a 4.0%/0.7% improvement of MAEt on ARKitFace/BIWI. |
| 源URL | [http://ir.ia.ac.cn/handle/173211/58560]  |
| 专题 | 自动化研究所_模式识别国家重点实验室_生物识别与安全技术研究中心 |
| 作者单位 | 1.Alibaba 2.CASIA |
| 推荐引用方式 GB/T 7714 | Xu M,Xiangyu Zhu,Yueying Kao,et al. Multi-Level Pixel-Wise Correspondence Learning for 6DoF Face Pose Estimation[J]. IEEE Transactions on Multimedia,2024:IEEE Xplore. |
| APA | Xu M,Xiangyu Zhu,Yueying Kao,Zhiwen Chen,Jiangjing Lyu,&Zhen Lei.(2024).Multi-Level Pixel-Wise Correspondence Learning for 6DoF Face Pose Estimation.IEEE Transactions on Multimedia,IEEE Xplore. |
| MLA | Xu M,et al."Multi-Level Pixel-Wise Correspondence Learning for 6DoF Face Pose Estimation".IEEE Transactions on Multimedia (2024):IEEE Xplore. |
入库方式： OAI收割
来源：自动化研究所
浏览0
下载0
收藏0
其他版本
除非特别说明，本系统中所有内容都受版权保护，并保留所有权利。