NFLB dropout: Improve generalization ability by dropping out the best -A biologically inspired adaptive dropout method for unsupervised learning
文献类型：会议论文
| 作者 | Peijie Yin ; Lu Qi; Xuanyang Xi; Bo Zhang; Hong Qiao
|
| 出版日期 | 2016 |
| 会议名称 | 2016 International Joint Conference on Neural Networks (IJCNN) |
| 会议日期 | 24-29 July 2016 |
| 会议地点 | Vancouver, BC, Canada |
| 关键词 | none |
| 通讯作者 | Peijie Yin |
| 英文摘要 | Generalization ability is widely acknowledged as one of the most important criteria to evaluate thequality of unsupervised models. The objective of our research is to find a better dropout method toimprove the generalization ability of convolutional deep belief network (CDBN), an unsupervised learningmodel for vision tasks. In this paper, the phenomenon of low feature diversity during the training process is investigated. The attention mechanism of human visual system is more focused on rare events and depresses well-known facts. Inspired by this mechanism, No Feature Left Behind Dropout (NFLBDropout), an adaptive dropout method is firstly proposed to automatically adjust the dropout rate feature-wisely. In the proposed method, the algorithm drops well-trained features and keeps poorly-trained ones with a high probability during training iterations. In addition, we apply two approximations of the quality of features, which are inspired by theory of saliency and optimization. Compared with themodel trained by standard dropout, experiment results show that our NFLB Dropout method improves not only the accuracy but the convergence speed as well. |
| 会议录 | 2016 International Joint Conference on Neural Networks (IJCNN)
 |
| 源URL | [http://ir.ia.ac.cn/handle/173211/12829]  |
| 专题 | 自动化研究所_复杂系统管理与控制国家重点实验室_机器人应用与理论组 |
| 推荐引用方式 GB/T 7714 | Peijie Yin,Lu Qi,Xuanyang Xi,et al. NFLB dropout: Improve generalization ability by dropping out the best -A biologically inspired adaptive dropout method for unsupervised learning[C]. 见:2016 International Joint Conference on Neural Networks (IJCNN). Vancouver, BC, Canada. 24-29 July 2016. |
入库方式： OAI收割
来源：自动化研究所
浏览0
下载0
收藏0
其他版本
除非特别说明，本系统中所有内容都受版权保护，并保留所有权利。