中国科学院机构知识库网格
Chinese Academy of Sciences Institutional Repositories Grid
首页 机构 成果 学者
  
  1. CAS IR Grid
  2. 自动化研究所
  3. 中国科学院自动化研究所
  4. 复杂系统认知与决策实验室
Real-time continuous detection and recognition of dynamic hand gestures in untrimmed sequences based on end-to-end architecture with 3D DenseNet and LSTM

文献类型：期刊论文

作者Lu, Zhi4; Qin, Shiyin2,3; Lv, Pin4; Sun, Liguo4; Tang, Bo1
刊名MULTIMEDIA TOOLS AND APPLICATIONS
出版日期2023-07-14
页码38
ISSN号1380-7501
关键词Continuous detection Gesture recognition Long short-term memory 3D densely connected convolutional networks Connectionist temporal classification Canonical correlation analysis
DOI10.1007/s11042-023-16130-1
通讯作者Lu, Zhi(zhi.lu@ia.ac.cn)
英文摘要With the continuous development of the deep learning theory, novel gesture recognition approaches have been constantly emerging, and the performance has also been continuously improved. However, most research methods focus on the recognition of isolated gestures, and the detection and recognition of continuous gestures are rarely studied. To this end, aiming at the real-time detection and classification of dynamic gestures in untrimmed sequences, a well-designed end-to-end architecture based on the variants of 3D DenseNet and unidirectional LSTM was hereby proposed as an effective tool to extract the discriminative spatio-temporal features of untrimmed hand gesture sequences. Then, connectionist temporal classification was combined to train the network on a publicly available dataset, and some effective capacities could be transferred to enhance the learning ability of the proposed network by training large gesture samples. In this way, the class-conditional probability of an incoming sequence belonging to a given gesture class was predicted and then compared with a predefined threshold to automatically determine the start and end of gestures. In addition, to enhance the classification accuracy of segmented gestures, a bidirectional LSTM network was utilized to model the temporal information, with both the past frames and the future ones taken into account. Finally, a continuous gesture dataset collected indoors for specific application was introduced to validate the proposed method. On this challenge dataset, the 3D DenseNet-LSTM model achieves real-time early detection and classification tasks on unsegmented gesture sequences, and the 3D DenseNet-BiLSTM not only achieves an accuracy of 92.06% on segmented gestures, but also a classification accuracy of 89.8% and 99.7% on nvGesture and SKIG public datasets, respectively. The experimental results demonstrate the performance advantages of the detection and classification as well as the real-time response speed.
WOS关键词FUSION ; ROBUST
资助项目National Natural Science Foundation of China[61731001] ; Natural Science Foundation of Zhejiang Province[LY21E050017]
WOS研究方向Computer Science ; Engineering
语种英语
出版者SPRINGER
WOS记录号WOS:001030536100003
资助机构National Natural Science Foundation of China ; Natural Science Foundation of Zhejiang Province
源URL[http://ir.ia.ac.cn/handle/173211/53726]  
专题复杂系统认知与决策实验室
通讯作者Lu, Zhi
作者单位1.China Jiliang Univ, Coll Metrol & Measurement Engn, Hangzhou 310018, Zhejiang, Peoples R China
2.Dongguan Univ Technol, Sch Elect Engn & Intelligentizat, Dongguan 523808, Peoples R China
3.Beihang Univ, Sch Automat Sci & Elect Engn, 37 Xueyuan Rd, Beijing 100191, Peoples R China
4.Chinese Acad Sci, Inst Automat, 95 Zhongguancun East Rd, Beijing 100190, Peoples R China
推荐引用方式
GB/T 7714		 Lu, Zhi,Qin, Shiyin,Lv, Pin,et al. Real-time continuous detection and recognition of dynamic hand gestures in untrimmed sequences based on end-to-end architecture with 3D DenseNet and LSTM[J]. MULTIMEDIA TOOLS AND APPLICATIONS,2023:38.
APA Lu, Zhi,Qin, Shiyin,Lv, Pin,Sun, Liguo,&Tang, Bo.(2023).Real-time continuous detection and recognition of dynamic hand gestures in untrimmed sequences based on end-to-end architecture with 3D DenseNet and LSTM.MULTIMEDIA TOOLS AND APPLICATIONS,38.
MLA Lu, Zhi,et al."Real-time continuous detection and recognition of dynamic hand gestures in untrimmed sequences based on end-to-end architecture with 3D DenseNet and LSTM".MULTIMEDIA TOOLS AND APPLICATIONS (2023):38.

入库方式： OAI收割

来源：自动化研究所

浏览0
下载0
收藏0
其他版本

除非特别说明，本系统中所有内容都受版权保护，并保留所有权利。