中国科学院机构知识库网格
Chinese Academy of Sciences Institutional Repositories Grid
首页 机构 成果 学者
  
  1. CAS IR Grid
  2. 自动化研究所
  3. 中国科学院自动化研究所
  4. 复杂系统认知与决策实验室
  5. 智能系统与工程
热门
View Adaptive Neural Networks for High Performance Skeleton-Based Human Action Recognition

文献类型：期刊论文

作者Zhang, Pengfei1; Lan, Cuiling3; Xing, Junliang5; Zeng, Wenjun4; Xue, Jianru2; Zheng, Nanning2
刊名IEEE TRANSACTIONS ON PATTERN ANALYSIS AND MACHINE INTELLIGENCE
出版日期2019-08-01
卷号41期号:8页码:1963-1978
关键词View adaptation skeleton action recognition RNN CNN consistent
ISSN号0162-8828
DOI10.1109/TPAMI.2019.2896631
通讯作者Lan, Cuiling(culan@microsoft.com) ; Xue, Jianru(jrxue@mail.xjtu.edu.cn)
英文摘要Skeleton-based human action recognition has recently attracted increasing attention thanks to the accessibility and the popularity of 3D skeleton data. One of the key challenges in action recognition lies in the large variations of action representations when they are captured from different viewpoints. In order to alleviate the effects of view variations, this paper introduces a novel view adaptation scheme, which automatically determines the virtual observation viewpoints over the course of an action in a learning based data driven manner. Instead of re-positioning the skeletons using a fixed human-defined prior criterion, we design two view adaptive neural networks, i.e., VA-RNN and VA-CNN, which are respectively built based on the recurrent neural network (RNN) with the Long Short-term Memory (LSTM) and the convolutional neural network (CNN). For each network, a novel view adaptation module learns and determines the most suitable observation viewpoints, and transforms the skeletons to those viewpoints for the end-to-end recognition with a main classification network. Ablation studies find that the proposed view adaptive models are capable of transforming the skeletons of various views to much more consistent virtual viewpoints. Therefore, the models largely eliminate the influence of the viewpoints, enabling the networks to focus on the learning of action-specific features and thus resulting in superior performance. In addition, we design a two-stream scheme (referred to as VA-fusion) that fuses the scores of the two networks to provide the final prediction, obtaining enhanced performance. Moreover, random rotation of skeleton sequences is employed to improve the robustness of view adaptation models and alleviate overfitting during training. Extensive experimental evaluations on five challenging benchmarks demonstrate the effectiveness of the proposed view-adaptive networks and superior performance over state-of-the-art approaches.
WOS关键词CLASSIFICATION ; MOTION ; ROBUST
资助项目National Key Research and Development Program of China[2016YFB1001004] ; Natural Science Foundation of China[61672519]
WOS研究方向Computer Science ; Engineering
语种英语
WOS记录号WOS:000473598800013
出版者IEEE COMPUTER SOC
资助机构National Key Research and Development Program of China ; Natural Science Foundation of China
源URL[http://ir.ia.ac.cn/handle/173211/26840]  
专题智能系统与工程
通讯作者Lan, Cuiling; Xue, Jianru
作者单位1.Xi An Jiao Tong Univ, Xian 710049, Shaanxi, Peoples R China
2.Xi An Jiao Tong Univ, Inst Artificial Intelligence & Robot, Xian 710049, Shaanxi, Peoples R China
3.Microsoft Res Asia, Beijing 100080, Peoples R China
4.Microsoft Res Asia, Senior Leadership Team, Beijing 100080, Peoples R China
5.Chinese Acad Sci, Inst Automat, Natl Lab Pattern Recognit, Beijing, Peoples R China
推荐引用方式
GB/T 7714		 Zhang, Pengfei,Lan, Cuiling,Xing, Junliang,et al. View Adaptive Neural Networks for High Performance Skeleton-Based Human Action Recognition[J]. IEEE TRANSACTIONS ON PATTERN ANALYSIS AND MACHINE INTELLIGENCE,2019,41(8):1963-1978.
APA Zhang, Pengfei,Lan, Cuiling,Xing, Junliang,Zeng, Wenjun,Xue, Jianru,&Zheng, Nanning.(2019).View Adaptive Neural Networks for High Performance Skeleton-Based Human Action Recognition.IEEE TRANSACTIONS ON PATTERN ANALYSIS AND MACHINE INTELLIGENCE,41(8),1963-1978.
MLA Zhang, Pengfei,et al."View Adaptive Neural Networks for High Performance Skeleton-Based Human Action Recognition".IEEE TRANSACTIONS ON PATTERN ANALYSIS AND MACHINE INTELLIGENCE 41.8(2019):1963-1978.

入库方式： OAI收割

来源：自动化研究所

浏览0
下载0
收藏0
其他版本

除非特别说明，本系统中所有内容都受版权保护，并保留所有权利。