Leveraging Joint-Action Embedding in Multiagent Reinforcement Learning for Cooperative Games
文献类型：期刊论文
| 作者 | Lou, Xingzhou1,2; Zhang, Junge1,2 ; Du, Yali3; Yu, Chao4; He, Zhaofeng5; Huang, Kaiqi1,2
|
| 刊名 | IEEE TRANSACTIONS ON GAMES
 |
| 出版日期 | 2024-06-01 |
| 卷号 | 16期号:2页码:470-482 |
| 关键词 | Games Predictive models Reinforcement learning Convergence Task analysis Correlation Training Joint-action embedding multiagent policy gradient reinforcement learning |
| ISSN号 | 2475-1502 |
| DOI | 10.1109/TG.2023.3302694 |
| 通讯作者 | Zhang, Junge(jgzhang@nlpr.ia.ac.cn) |
| 英文摘要 | State-of-the-art multiagent policy gradient (MAPG) methods have demonstrated convincing capability in many cooperative games. However, the exponentially growing joint-action space severely challenges the critic's value evaluation and hinders performance of MAPG methods. To address this issue, we augment Central-Q policy gradient with a joint-action embedding function and propose mutual-information maximization MAPG (M3APG). The joint-action embedding function makes joint-actions contain information of state transitions, which will improve the critic's generalization over the joint-action space by allowing it to infer joint-actions' outcomes. We theoretically prove that with a fixed joint-action embedding function, the convergence of M3APG is guaranteed. Experiment results of the StarCraft multiagent challenge (SMAC) demonstrate that M3APG gives evaluation results with better accuracy and outperform other MAPG basic models across various maps of multiple difficulty levels. We empirically show that our joint-action embedding model can be extended to value-based multiagent reinforcement learning methods and state-of-the-art MAPG methods. Finally, we run an ablation study to show that the usage of mutual information in our method is necessary and effective. |
| WOS关键词 | LEVEL |
| 资助项目 | Basic Cultivation Fund Project |
| WOS研究方向 | Computer Science |
| 语种 | 英语 |
| WOS记录号 | WOS:001252777700008 |
| 出版者 | IEEE-INST ELECTRICAL ELECTRONICS ENGINEERS INC |
| 资助机构 | Basic Cultivation Fund Project |
| 源URL | [http://ir.ia.ac.cn/handle/173211/59106]  |
| 专题 | 智能系统与工程 |
| 通讯作者 | Zhang, Junge |
| 作者单位 | 1.Chinese Acad Sci, Univ Chinese Acad Sci, Sch Artificial Intelligence, Beijing 100190, Peoples R China 2.Chinese Acad Sci, Inst Automat, Ctr Res Intelligent Syst & Engn, Beijing 100190, Peoples R China 3.Kings Coll London, Dept Informat, London WC2R2LS, England 4.Sun Yat Sen Univ, Sch Data & Comp Sci, Guangzhou 510275, Peoples R China 5.Beijing Univ Posts & Telecommun, Sch Artificial Intelligence, Beijing 100876, Peoples R China |
| 推荐引用方式 GB/T 7714 | Lou, Xingzhou,Zhang, Junge,Du, Yali,et al. Leveraging Joint-Action Embedding in Multiagent Reinforcement Learning for Cooperative Games[J]. IEEE TRANSACTIONS ON GAMES,2024,16(2):470-482. |
| APA | Lou, Xingzhou,Zhang, Junge,Du, Yali,Yu, Chao,He, Zhaofeng,&Huang, Kaiqi.(2024).Leveraging Joint-Action Embedding in Multiagent Reinforcement Learning for Cooperative Games.IEEE TRANSACTIONS ON GAMES,16(2),470-482. |
| MLA | Lou, Xingzhou,et al."Leveraging Joint-Action Embedding in Multiagent Reinforcement Learning for Cooperative Games".IEEE TRANSACTIONS ON GAMES 16.2(2024):470-482. |
入库方式： OAI收割
来源：自动化研究所
浏览0
下载0
收藏0
其他版本
除非特别说明，本系统中所有内容都受版权保护，并保留所有权利。