Pruning SMAC search space based on key hyperparameters
文献类型：期刊论文
| 作者 | Li, Hui1,4; Liang, Qingqing1,4; Chen, Mei1,4; Dai, Zhenyu1,4; Li, Huanjun2; Zhu, Ming3 |
| 刊名 | CONCURRENCY AND COMPUTATION-PRACTICE & EXPERIENCE
 |
| 出版日期 | 2020-06-01 |
| 页码 | 11 |
| ISSN号 | 1532-0626 |
| 关键词 | hyperparameter optimization SMAC pruning key hyperparameters AutoML |
| DOI | 10.1002/cpe.5805 |
| 英文摘要 | Machine learning (ML) has been widely applied in many areas in recent decades. However, because of its inherent complexity and characteristics, the efficiency and effectiveness of ML algorithm often to be heavily relies on the technical experts' experience and expertise which play a crucial role to optimize hyperparameters of algorithms. Generally, the procedure tuning the exposed hyperparameters of ML algorithm to achieve better performance is called Hyperparameters Optimization. Traditional hyperparameters optimization methods are manually exhaustive search, which is unavailable for high dimensional search spaces and large datasets. Recent automated sequential model-based optimization led to substantial improvements for this problem, whose core idea is fitting a regression model to describe the importance and dependence of algorithm's performance on certain given hyperparameter setting. Sequential model-based algorithm configuration (SMAC) is a the-state-of-art approach, which is specified by four components, Initialize, FitModel, SelectConfigurations, and Intensify. In this article, we propose to add a pruning procedure into SMAC approach, it quantifies the importance of hyperparameters by analyzing the performance of a list of promising configurations and reduces search space by discarding noncritical and bad key hyperparameters. To investigate the impact of pruning for model's performance, we conducted experiments on the configuration space constructed by Auto-Sklearn and compared the effect of run time and pruning ratio with our algorithm. The experiments results verified that, our method made the configuration selected by SMAC more stable and achieved better performance. |
| 资助项目 | National Natural Science Foundation of China[61462012] ; National Natural Science Foundation of China[61562010] ; National Natural Science Foundation of China[U1531246] ; National Natural Science Foundation of China[71964009] ; Innovation Team of the Data Analysis Cloud Service of Guizhou Province[[2015]53] ; Program for Innovative Talent of Guizhou Province |
| WOS研究方向 | Computer Science |
| 语种 | 英语 |
| 出版者 | WILEY |
| WOS记录号 | WOS:000536519000001 |
| 资助机构 | National Natural Science Foundation of China ; National Natural Science Foundation of China ; Innovation Team of the Data Analysis Cloud Service of Guizhou Province ; Innovation Team of the Data Analysis Cloud Service of Guizhou Province ; Program for Innovative Talent of Guizhou Province ; Program for Innovative Talent of Guizhou Province ; National Natural Science Foundation of China ; National Natural Science Foundation of China ; Innovation Team of the Data Analysis Cloud Service of Guizhou Province ; Innovation Team of the Data Analysis Cloud Service of Guizhou Province ; Program for Innovative Talent of Guizhou Province ; Program for Innovative Talent of Guizhou Province ; National Natural Science Foundation of China ; National Natural Science Foundation of China ; Innovation Team of the Data Analysis Cloud Service of Guizhou Province ; Innovation Team of the Data Analysis Cloud Service of Guizhou Province ; Program for Innovative Talent of Guizhou Province ; Program for Innovative Talent of Guizhou Province ; National Natural Science Foundation of China ; National Natural Science Foundation of China ; Innovation Team of the Data Analysis Cloud Service of Guizhou Province ; Innovation Team of the Data Analysis Cloud Service of Guizhou Province ; Program for Innovative Talent of Guizhou Province ; Program for Innovative Talent of Guizhou Province |
| 源URL | [http://ir.bao.ac.cn/handle/114a11/54615]  |
| 专题 | 中国科学院国家天文台 |
| 通讯作者 | Chen, Mei |
| 作者单位 | 1.Guizhou Univ, Guizhou Engineer Lab ACMIS, Guiyang, Peoples R China 2.Aerosp Jiangnan Grp Co Ltd, Guiyang, Peoples R China 3.Chinese Acad Sci, Natl Astron Observ, Beijing, Peoples R China 4.Guizhou Univ, Coll Comp Sci & Technol, Guiyang 550025, Peoples R China |
| 推荐引用方式 GB/T 7714 | Li, Hui,Liang, Qingqing,Chen, Mei,et al. Pruning SMAC search space based on key hyperparameters[J]. CONCURRENCY AND COMPUTATION-PRACTICE & EXPERIENCE,2020:11. |
| APA | Li, Hui,Liang, Qingqing,Chen, Mei,Dai, Zhenyu,Li, Huanjun,&Zhu, Ming.(2020).Pruning SMAC search space based on key hyperparameters.CONCURRENCY AND COMPUTATION-PRACTICE & EXPERIENCE,11. |
| MLA | Li, Hui,et al."Pruning SMAC search space based on key hyperparameters".CONCURRENCY AND COMPUTATION-PRACTICE & EXPERIENCE (2020):11. |
入库方式： OAI收割
来源：国家天文台
浏览0
下载0
收藏0
其他版本
除非特别说明，本系统中所有内容都受版权保护，并保留所有权利。