中国科学院机构知识库网格
Chinese Academy of Sciences Institutional Repositories Grid
首页 机构 成果 学者
  
  1. CAS IR Grid
  2. 自动化研究所
  3. 中国科学院自动化研究所
  4. 复杂系统认知与决策实验室
  5. 智能系统与工程
Multi-Modality Multi-Task Recurrent Neural Network for Online Action Detection

文献类型：期刊论文

作者Liu, Jiaying1; Li, Yanghao1; Song, Sijie1; Xing, Junliang2; Lan, Cuiling3; Zeng, Wenjun3
刊名IEEE TRANSACTIONS ON CIRCUITS AND SYSTEMS FOR VIDEO TECHNOLOGY
出版日期2019-09-01
卷号29期号:9页码:2667-2682
ISSN号1051-8215
关键词Action detection recurrent neural network multi-modality joint classification-regression
DOI10.1109/TCSVT.2018.2799968
通讯作者Liu, Jiaying(liujiaying@pku.edu.cn)
英文摘要Online action detection is a brand new challenge and plays a critical role in visual surveillance analytics. It goes one step further than a conventional action recognition task, which recognizes human actions from well-segmented clips. Online action detection is desired to identify the action type and localize action positions on the fly from the untrimmed stream data. In this paper, we propose a multi-modality multi-task recurrent neural network, which incorporates both RGB and Skeleton networks. We design different temporal modeling networks to capture specific characteristics from various modalities. Then, a deep long short-term memory subnetwork is utilized effectively to capture the complex long-range temporal dynamics, naturally avoiding the conventional sliding window design and thus ensuring high computational efficiency. Constrained by a multi-task objective function in the training phase, this network achieves superior detection performance and is capable of automatically localizing the start and end points of actions more accurately. Furthermore, embedding subtask of regression provides the ability to forecast the action prior to its occurrence. We evaluate the proposed method and several other methods in action detection and forecasting on the online action detection data set and gaming action data set datasets. Experimental results demonstrate that our model achieves the state-of-the-art performance on both tasks.
WOS关键词ACTION RECOGNITION ; ENSEMBLE ; MOTION
资助项目NVIDIA Corporation
WOS研究方向Engineering
语种英语
出版者IEEE-INST ELECTRICAL ELECTRONICS ENGINEERS INC
WOS记录号WOS:000489738900012
资助机构NVIDIA Corporation
源URL[http://ir.ia.ac.cn/handle/173211/26635]  
专题智能系统与工程
通讯作者Liu, Jiaying
作者单位1.Peking Univ, Inst Comp Sci & Technol, Beijing 100080, Peoples R China
2.Chinese Acad Sci, Inst Automat, Beijing 100080, Peoples R China
3.Microsoft Res Asia, Beijing 100080, Peoples R China
推荐引用方式
GB/T 7714		 Liu, Jiaying,Li, Yanghao,Song, Sijie,et al. Multi-Modality Multi-Task Recurrent Neural Network for Online Action Detection[J]. IEEE TRANSACTIONS ON CIRCUITS AND SYSTEMS FOR VIDEO TECHNOLOGY,2019,29(9):2667-2682.
APA Liu, Jiaying,Li, Yanghao,Song, Sijie,Xing, Junliang,Lan, Cuiling,&Zeng, Wenjun.(2019).Multi-Modality Multi-Task Recurrent Neural Network for Online Action Detection.IEEE TRANSACTIONS ON CIRCUITS AND SYSTEMS FOR VIDEO TECHNOLOGY,29(9),2667-2682.
MLA Liu, Jiaying,et al."Multi-Modality Multi-Task Recurrent Neural Network for Online Action Detection".IEEE TRANSACTIONS ON CIRCUITS AND SYSTEMS FOR VIDEO TECHNOLOGY 29.9(2019):2667-2682.

入库方式： OAI收割

来源：自动化研究所

浏览0
下载0
收藏0
其他版本

除非特别说明，本系统中所有内容都受版权保护，并保留所有权利。