Trigonometric Inference Providing Learning in Deep Neural Networks
文献类型：期刊论文
| 作者 | Cai, Jingyong2; Takemoto, Masashi3; Qiu, Yuming1 ; Nakajo, Hironori2
|
| 刊名 | APPLIED SCIENCES-BASEL
 |
| 出版日期 | 2021-08-01 |
| 卷号 | 11期号:15页码:15 |
| 关键词 | backpropagation neural network training trigonometric inference |
| DOI | 10.3390/app11156704 |
| 通讯作者 | Nakajo, Hironori(nakajo@cc.tuat.ac.jp) |
| 英文摘要 | Despite being heavily used in the training of deep neural networks (DNNs), multipliers are resource-intensive and insufficient in many different scenarios. Previous discoveries have revealed the superiority when activation functions, such as the sigmoid, are calculated by shift-and-add operations, although they fail to remove multiplications in training altogether. In this paper, we propose an innovative approach that can convert all multiplications in the forward and backward inferences of DNNs into shift-and-add operations. Because the model parameters and backpropagated errors of a large DNN model are typically clustered around zero, these values can be approximated by their sine values. Multiplications between the weights and error signals are transferred to multiplications of their sine values, which are replaceable with simpler operations with the help of the product to sum formula. In addition, a rectified sine activation function is utilized for further converting layer inputs into sine values. In this way, the original multiplication-intensive operations can be computed through simple add-and-shift operations. This trigonometric approximation method provides an efficient training and inference alternative for devices with insufficient hardware multipliers. Experimental results demonstrate that this method is able to obtain a performance close to that of classical training algorithms. The approach we propose sheds new light on future hardware customization research for machine learning. |
| 资助项目 | New Energy and Industrial Technology Development Organization (NEDO)[JPNP16007] ; JSPS KAKENHI[21K11804] ; JSPS KAKENHI[19K11879] |
| WOS研究方向 | Chemistry ; Engineering ; Materials Science ; Physics |
| 语种 | 英语 |
| WOS记录号 | WOS:000681990600001 |
| 出版者 | MDPI |
| 源URL | [http://119.78.100.138/handle/2HOD01W0/13873]  |
| 专题 | 中国科学院重庆绿色智能技术研究院 |
| 通讯作者 | Nakajo, Hironori |
| 作者单位 | 1.Chinese Acad Sci, Chongqing Inst Green & Intelligent Technol, Chongqing 400010, Peoples R China 2.Tokyo Univ Agr & Technol, Inst Engn, Tokyo 1848588, Japan 3.BeatCraft Inc, Tokyo 1300013, Japan |
| 推荐引用方式 GB/T 7714 | Cai, Jingyong,Takemoto, Masashi,Qiu, Yuming,et al. Trigonometric Inference Providing Learning in Deep Neural Networks[J]. APPLIED SCIENCES-BASEL,2021,11(15):15. |
| APA | Cai, Jingyong,Takemoto, Masashi,Qiu, Yuming,&Nakajo, Hironori.(2021).Trigonometric Inference Providing Learning in Deep Neural Networks.APPLIED SCIENCES-BASEL,11(15),15. |
| MLA | Cai, Jingyong,et al."Trigonometric Inference Providing Learning in Deep Neural Networks".APPLIED SCIENCES-BASEL 11.15(2021):15. |
入库方式： OAI收割
来源：重庆绿色智能技术研究院
浏览0
下载0
收藏0
其他版本
除非特别说明，本系统中所有内容都受版权保护，并保留所有权利。