A Two-Stream CNN Framework for American Sign Language Recognition Based on Multimodal Data Fusion
文献类型：会议论文
| 作者 | Gao Q(高庆)1,3,4 ; Ogenyi, Uchenna Emeoha2; Liu JG(刘金国)1,3; Liu, Honghai2
|
| 出版日期 | 2019 |
| 会议日期 | September 4, 2019 - September 6, 2019 |
| 会议地点 | Portsmouth, United kingdom |
| 关键词 | Hand gesture recognition CNN Multimodal data fusion |
| 页码 | 107-118 |
| 英文摘要 | At present, vision-based hand gesture recognition is very important in human-robot interaction (HRI). This non-contact method enables natural and friendly interaction between people and robots. Aiming at this technology, a two-stream CNN framework (2S-CNN) is proposed to recognize the American sign language (ASL) hand gestures based on multimodal (RGB and depth) data fusion. Firstly, the hand gesture data is enhanced to remove the influence of background and noise. Secondly, hand gesture RGB and depth features are extracted for hand gesture recognition using CNNs on two streams, respectively. Finally, a fusion layer is designed for fusing the recognition results of the two streams. This method utilizes multimodal data to increase the recognition accuracy of the ASL hand gestures. The experiments prove that the recognition accuracy of 2S-CNN can reach 92.08 $$\%$$ on ASL fingerspelling database and is higher than that of baseline methods. |
| 产权排序 | 1 |
| 会议录 | Advances in Computational Intelligence Systems - Contributions Presented at the 19th UK Workshop on Computational Intelligence, 2019
 |
| 会议录出版者 | Springer Verlag |
| 语种 | 英语 |
| ISSN号 | 2194-5357 |
| ISBN号 | 978-3-030-29932-3 |
| WOS记录号 | WOS:000618179900009 |
| 源URL | [http://ir.sia.cn/handle/173321/25741]  |
| 专题 | 沈阳自动化研究所_空间自动化技术研究室 |
| 通讯作者 | Liu JG(刘金国) |
| 作者单位 | 1.State Key Laboratory of Robotics, Shenyang Institute of Automation, Chinese Academy of Sciences, Shenyang 110016, China 2.School of Computing, University of Portsmouth, Portsmouth 3.Institutes for Robotics and Intelligent Manufacturing, Chinese Academy of Sciences, Shenyang 110169, China 4.University of Chinese Academy of Sciences, Beijing 100049, China 5.PO1 3HE, United Kingdom |
| 推荐引用方式 GB/T 7714 | Gao Q,Ogenyi, Uchenna Emeoha,Liu JG,et al. A Two-Stream CNN Framework for American Sign Language Recognition Based on Multimodal Data Fusion[C]. 见:. Portsmouth, United kingdom. September 4, 2019 - September 6, 2019. |
入库方式： OAI收割
来源：沈阳自动化研究所
浏览0
下载0
收藏0
其他版本
除非特别说明，本系统中所有内容都受版权保护，并保留所有权利。