中国科学院机构知识库网格
Chinese Academy of Sciences Institutional Repositories Grid
首页 机构 成果 学者
  
  1. CAS IR Grid
  2. 自动化研究所
  3. 中国科学院自动化研究所
  4. 脑图谱与类脑智能实验室
  5. 脑网络组研究中心
Joint Learning in the Spatio-Temporal and Frequency Domains for Skeleton-Based Action Recognition

文献类型：期刊论文

作者Hu, Guyue1,3,4; Cui, Bo1,3,4; Yu, Shan1,2,3,4; Guyue, Hu
刊名IEEE TRANSACTIONS ON MULTIMEDIA
出版日期2020-09-01
卷号22期号:9页码:2207-2220
关键词Frequency-domain analysis Transforms Frequency synchronization Semantics Training Skeleton Data mining Action recognition frequency attention synchronous local and non-local learning soft-margin focal loss multi-task learning
ISSN号1520-9210
DOI10.1109/TMM.2019.2953325
通讯作者Hu, Guyue(guyue.hu@nlpr.ia.ac.cn)
英文摘要Benefiting from its succinctness and robustness, skeleton-based action recognition has recently attracted much attention. Most existing methods utilize local networks (e.g. recurrent network, convolutional network, and graph convolutional network) to extract spatio-temporal dynamics hierarchically. As a consequence, the local and non-local dependencies, which contain more details and semantics respectively, are asynchronously captured in different level of layers. Moreover, existing methods are limited to the spatio-temporal domain and ignore information in the frequency domain. To better extract synchronous detailed and semantic information from multi-domains, we propose a residual frequency attention (rFA) block to focus on discriminative patterns in the frequency domain, and a synchronous local and non-local (SLnL) block to simultaneously capture the details and semantics in the spatio-temporal domain. In addition, to optimize the whole learning processes of the multi-branch network, we put it under a pseudo multi-task learning paradigm. During training, 1) a soft-margin focal loss (SMFL) is proposed to optimize the intra-branch separated learning process, which can automatically conduct data selection and encourage intrinsic margins in classifiers; 2) A mutual learning policy is also proposed to further facilitate the inter-branch collaborative learning process. Eventually, our approach achieves the state-of-the-art performance on several large-scale datasets for skeleton-based action recognition.
WOS关键词ENSEMBLE
资助项目National Key Research and Development Program of China[2017YFA0105203] ; Strategic Priority Research Program of the Chinese Academy of Sciences (CAS)[XDB32040200] ; Hundred-Talent Program of CAS
WOS研究方向Computer Science ; Telecommunications
语种英语
WOS记录号WOS:000562310200002
出版者IEEE-INST ELECTRICAL ELECTRONICS ENGINEERS INC
资助机构National Key Research and Development Program of China ; Strategic Priority Research Program of the Chinese Academy of Sciences (CAS) ; Hundred-Talent Program of CAS
源URL[http://ir.ia.ac.cn/handle/173211/40515]  
专题自动化研究所_脑网络组研究中心
通讯作者Hu, Guyue; Guyue, Hu
作者单位1.Chinese Acad Sci, Brainnetome Ctr, Inst Automat, Beijing 100190, Peoples R China
2.CAS Ctr Excellence Brain Sci & Intelligence Techn, Beijing 100190, Peoples R China
3.Univ Chinese Acad Sci, Beijing 100049, Peoples R China
4.Chinese Acad Sci, Natl Labo Pattern Recognit, Inst Automat, Beijing 100190, Peoples R China
推荐引用方式
GB/T 7714		 Hu, Guyue,Cui, Bo,Yu, Shan,et al. Joint Learning in the Spatio-Temporal and Frequency Domains for Skeleton-Based Action Recognition[J]. IEEE TRANSACTIONS ON MULTIMEDIA,2020,22(9):2207-2220.
APA Hu, Guyue,Cui, Bo,Yu, Shan,&Guyue, Hu.(2020).Joint Learning in the Spatio-Temporal and Frequency Domains for Skeleton-Based Action Recognition.IEEE TRANSACTIONS ON MULTIMEDIA,22(9),2207-2220.
MLA Hu, Guyue,et al."Joint Learning in the Spatio-Temporal and Frequency Domains for Skeleton-Based Action Recognition".IEEE TRANSACTIONS ON MULTIMEDIA 22.9(2020):2207-2220.

入库方式： OAI收割

来源：自动化研究所

浏览0
下载0
收藏0
其他版本

除非特别说明，本系统中所有内容都受版权保护，并保留所有权利。