MixGradient: A gradient-based re-weighting scheme with mixup for imbalanced data streams
文献类型：期刊论文
| 作者 | Peng, Xinyu1; Wang, Fei-Yue3 ; Li, Li1,2
|
| 刊名 | NEURAL NETWORKS
 |
| 出版日期 | 2023-04-01 |
| 卷号 | 161页码:525-534 |
| 关键词 | Deep learning Imbalanced data streams Sample gradient Typical samples Mixup |
| ISSN号 | 0893-6080 |
| DOI | 10.1016/j.neunet.2023.02.017 |
| 通讯作者 | Li, Li(li-li@tsinghua.edu.cn) |
| 英文摘要 | A challenge for contemporary deep neural networks in real-world problems is learning from an imbalanced data stream, where data tends to be received chunk by chunk over time, and the prior class distribution is severely imbalanced. Although many sophisticated algorithms have been derived, most of them overlook the importance of gradient information. From this perspective, the difficulty of learning from imbalanced data streams lies in the fact that the gradient estimated on an uneven class distribution is not informative enough to reflect the critical pattern of each class. To this end, we propose to assign higher weights on the training samples whose gradients are close to the gradient of corresponding typical samples, thus highlighting the important samples in minority classes and suppressing the noisy samples in majority classes. Such an idea can be combined with Mixup, which exploits the interpolation information of data to further compensate for the information of sample space that the typical samples do not provide and expand the role of the proposed re -weighting scheme. Experiments on artificially induced long-tailed CIFAR data streams and long-tailed MiniPlaces data stream show that the resulting method, termed MixGradient, boosts the generalization performance of DNNs under different imbalance ratios and achieves up to 10% accuracy improvement.(c) 2023 Elsevier Ltd. All rights reserved. |
| WOS关键词 | EXTREME LEARNING-MACHINE ; NEURAL-NETWORKS |
| 资助项目 | National Key Research and Development Program of China[2020AAA0108104] |
| WOS研究方向 | Computer Science ; Neurosciences & Neurology |
| 语种 | 英语 |
| WOS记录号 | WOS:000990516400001 |
| 出版者 | PERGAMON-ELSEVIER SCIENCE LTD |
| 资助机构 | National Key Research and Development Program of China |
| 源URL | [http://ir.ia.ac.cn/handle/173211/53311]  |
| 专题 | 多模态人工智能系统全国重点实验室 |
| 通讯作者 | Li, Li |
| 作者单位 | 1.Tsinghua Univ, Dept Automat, Beijing 100084, Peoples R China 2.Tsinghua Univ, Natl Lab Informat Sci & Technol, Beijing 100084, Peoples R China 3.Chinese Acad Sci, Inst Automat, State Key Lab Management & Control Complex Syst, Beijing 100080, Peoples R China |
| 推荐引用方式 GB/T 7714 | Peng, Xinyu,Wang, Fei-Yue,Li, Li. MixGradient: A gradient-based re-weighting scheme with mixup for imbalanced data streams[J]. NEURAL NETWORKS,2023,161:525-534. |
| APA | Peng, Xinyu,Wang, Fei-Yue,&Li, Li.(2023).MixGradient: A gradient-based re-weighting scheme with mixup for imbalanced data streams.NEURAL NETWORKS,161,525-534. |
| MLA | Peng, Xinyu,et al."MixGradient: A gradient-based re-weighting scheme with mixup for imbalanced data streams".NEURAL NETWORKS 161(2023):525-534. |
入库方式： OAI收割
来源：自动化研究所
浏览0
下载0
收藏0
其他版本
除非特别说明，本系统中所有内容都受版权保护，并保留所有权利。