中国科学院机构知识库网格
Chinese Academy of Sciences Institutional Repositories Grid
首页 机构 成果 学者
  
  1. CAS IR Grid
  2. 自动化研究所
  3. 中国科学院自动化研究所
  4. 多模态人工智能系统全国重点实验室
PyMAF-X: Towards Well-Aligned Full-Body Model Regression From Monocular Images

文献类型：期刊论文

作者Zhang, Hongwen2; Tian, Yating3; Zhang, Yuxiang2; Li, Mengcheng2; An, Liang2; Sun, Zhenan1; Liu, Yebin2
刊名IEEE TRANSACTIONS ON PATTERN ANALYSIS AND MACHINE INTELLIGENCE
出版日期2023-10-01
卷号45期号:10页码:12287-12303
ISSN号0162-8828
关键词Expressive human mesh recovery full-body motion capture mesh alignment feedback monocular 3D reconstruction
DOI10.1109/TPAMI.2023.3271691
通讯作者Liu, Yebin(liuyebin@mail.tsinghua.edu.cn)
英文摘要We present PyMAF-X, a regression-based approach to recovering a parametric full-body model from a single image. This task is very challenging since minor parametric deviation may lead to noticeable misalignment between the estimated mesh and the input image. Moreover, when integrating part-specific estimations into the full-body model, existing solutions tend to either degrade the alignment or produce unnatural wrist poses. To address these issues, we propose a Pyramidal Mesh Alignment Feedback (PyMAF) loop in our regression network for well-aligned human mesh recovery and extend it as PyMAF-X for the recovery of expressive full-body models. The core idea of PyMAF is to leverage a feature pyramid and rectify the predicted parameters explicitly based on the mesh-image alignment status. Specifically, given the currently predicted parameters, mesh-aligned evidence will be extracted from finer-resolution features accordingly and fed back for parameter rectification. To enhance the alignment perception, an auxiliary dense supervision is employed to provide mesh-image correspondence guidance while spatial alignment attention is introduced to enable the awareness of the global contexts for our network. When extending PyMAF for full-body mesh recovery, an adaptive integration strategy is proposed in PyMAF-X to produce natural wrist poses while maintaining the well-aligned performance of the part-specific estimations. The efficacy of our approach is validated on several benchmark datasets for body, hand, face, and full-body mesh recovery, where PyMAF and PyMAF-X effectively improve the mesh-image alignment and achieve new The project page with code and video results can be found at https://www.liuyebin.com/pymaf-x.
WOS关键词HUMAN SHAPE ; POSE ; REPRESENTATION
资助项目National Key Ramp;D Program of China ; National Natural Science Foundation of China[2021ZD0113501] ; National Natural Science Foundation of China[62125107] ; National Natural Science Foundation of China[U1836217] ; China Postdoctoral Science Foundation[62276263] ; [2022M721844]
WOS研究方向Computer Science ; Engineering
语种英语
出版者IEEE COMPUTER SOC
WOS记录号WOS:001068816800049
资助机构National Key Ramp;D Program of China ; National Natural Science Foundation of China ; China Postdoctoral Science Foundation
源URL[http://ir.ia.ac.cn/handle/173211/53047]  
专题多模态人工智能系统全国重点实验室
通讯作者Liu, Yebin
作者单位1.Chinese Acad Sci, Inst Automat, Beijing 100190, Peoples R China
2.Tsinghua Univ, Dept Automat, Beijing 100084, Peoples R China
3.Nanjing Univ, Dept Comp Sci & Technol, Nanjing 210023, Peoples R China
推荐引用方式
GB/T 7714		 Zhang, Hongwen,Tian, Yating,Zhang, Yuxiang,et al. PyMAF-X: Towards Well-Aligned Full-Body Model Regression From Monocular Images[J]. IEEE TRANSACTIONS ON PATTERN ANALYSIS AND MACHINE INTELLIGENCE,2023,45(10):12287-12303.
APA Zhang, Hongwen.,Tian, Yating.,Zhang, Yuxiang.,Li, Mengcheng.,An, Liang.,...&Liu, Yebin.(2023).PyMAF-X: Towards Well-Aligned Full-Body Model Regression From Monocular Images.IEEE TRANSACTIONS ON PATTERN ANALYSIS AND MACHINE INTELLIGENCE,45(10),12287-12303.
MLA Zhang, Hongwen,et al."PyMAF-X: Towards Well-Aligned Full-Body Model Regression From Monocular Images".IEEE TRANSACTIONS ON PATTERN ANALYSIS AND MACHINE INTELLIGENCE 45.10(2023):12287-12303.

入库方式： OAI收割

来源：自动化研究所

浏览0
下载0
收藏0
其他版本

除非特别说明，本系统中所有内容都受版权保护，并保留所有权利。