Deep representation learning for domain generalization with information bottleneck principle
文献类型：期刊论文
| 作者 | Zhang, Jiao1,2 ; Zhang, Xu-Yao1,2; Wang, Chuang1,2; Liu, Cheng-Lin1,2
|
| 刊名 | PATTERN RECOGNITION
 |
| 出版日期 | 2023-11-01 |
| 卷号 | 143页码:12 |
| ISSN号 | 0031-3203 |
| 关键词 | Domain generalization Information bottleneck Representation learning |
| DOI | 10.1016/j.patcog.2023.109737 |
| 通讯作者 | Zhang, Jiao(zhangjiao2019@ia.ac.cn) |
| 英文摘要 | Although deep neural networks have achieved superior performance on many classical tasks, they deteri-orate in real applications due to the unpredictable distribution shift. Domain generalization (DG) focuses on improving the generalization ability of the predictive model in unseen domains by training on multi-ple available source domains. All these domains share the same categories but commonly obey different distributions. In this paper, we establish a new theoretical framework for domain generalization from the perspective of the information bottleneck (IB) principle, which links representation learning in DG with domain-invariant representation learning and maximizing feature entropy (MFE). Based on the the-oretical framework, we provide a feasible solution by class-wise instance discrimination combined with inter-dimension decorrelation and intra-dimension uniformity to learn the desired representation for do-main generalization, which achieves excellent performance on multiple datasets without knowing domain labels. Extensive experiments show that the proposed regularization rule (MFE) can improve invariance -based DG methods consistently. Moreover, as an extreme case of domain generalization, we also show that MFE is promising to improve adversarial robustness. (c) 2023 Elsevier Ltd. All rights reserved. |
| 资助项目 | National Key Research and Development Program[2018AAA010 040 0] ; National Natural Science Foundation of China (NSFC)[U20A20223] ; National Natural Science Foundation of China (NSFC)[62222609] ; National Natural Science Foundation of China (NSFC)[61721004] ; National Natural Science Foundation of China (NSFC)[62076236] ; Youth Innovation Promotion Association of Chinese Academy of Sciences[2019141] ; Key Research Program of Frontier Sciences of Chinese Academy of Sciences[ZDBS-LY-7004] ; Pioneer Hundred Talents Program of CAS[Y9S9MS08] |
| WOS研究方向 | Computer Science ; Engineering |
| 语种 | 英语 |
| 出版者 | ELSEVIER SCI LTD |
| WOS记录号 | WOS:001017964900001 |
| 资助机构 | National Key Research and Development Program ; National Natural Science Foundation of China (NSFC) ; Youth Innovation Promotion Association of Chinese Academy of Sciences ; Key Research Program of Frontier Sciences of Chinese Academy of Sciences ; Pioneer Hundred Talents Program of CAS |
| 源URL | [http://ir.ia.ac.cn/handle/173211/53720]  |
| 专题 | 多模态人工智能系统全国重点实验室 |
| 通讯作者 | Zhang, Jiao |
| 作者单位 | 1.Chinese Acad Sci, Natl Lab Pattern Recognit NLPR, Inst Automat, Beijing 100190, Peoples R China 2.Univ Chinese Acad Sci UCAS, Sch Artificial Intelligence, Beijing 100049, Peoples R China |
| 推荐引用方式 GB/T 7714 | Zhang, Jiao,Zhang, Xu-Yao,Wang, Chuang,et al. Deep representation learning for domain generalization with information bottleneck principle[J]. PATTERN RECOGNITION,2023,143:12. |
| APA | Zhang, Jiao,Zhang, Xu-Yao,Wang, Chuang,&Liu, Cheng-Lin.(2023).Deep representation learning for domain generalization with information bottleneck principle.PATTERN RECOGNITION,143,12. |
| MLA | Zhang, Jiao,et al."Deep representation learning for domain generalization with information bottleneck principle".PATTERN RECOGNITION 143(2023):12. |
入库方式： OAI收割
来源：自动化研究所
浏览0
下载0
收藏0
其他版本
除非特别说明，本系统中所有内容都受版权保护，并保留所有权利。