Multiexperience-Assisted Efficient Multiagent Reinforcement Learning
文献类型:期刊论文
作者 | Zhang TL(张天乐)2,3; Liu Z(刘振)2,3; Yi JQ(易建强)2,3; Wu SG(吴士广)2,3; Pu ZQ(蒲志强)2,3; Zhao YJ(赵彦杰)1 |
刊名 | IEEE Transactions on Neural Networks and Learning Systems |
出版日期 | 2023 |
页码 | 1-15 |
文献子类 | 研究性论文 |
英文摘要 | Recently, multiagent reinforcement learning (MARL) has shown great potential for learning cooperative policies in multiagent systems (MASs). However, a noticeable drawback of current MARL is the low sample efficiency, which causes a huge amount of interactions with environment. Such amount of interactions greatly hinders the real-world application of MARL. Fortunately, effectively incorporating experience knowledge can assist MARL to quickly find effective solutions, which can significantly alleviate the drawback. In this article, a novel multiexperience-assisted reinforcement learning (MEARL) method is proposed to improve the learning efficiency of MASs. Specifically, monotonicity-constrained reward shaping is innovatively designed using expert experience to provide additional individual rewards to guide multiagent learning efficiently, with the invariance guarantee of the team optimization objective. Furthermore, a reward distribution estimator is specially developed to model an implicated reward distribution of environment by using transition experience from environment, containing collected samples (state–action pair, reward, and next state). This estimator can predict the expectation reward of each agent for the taken action to accurately estimate the state value function and accelerate its convergence. Besides, the performance of MEARL is evaluated on two multiagent environment plat- forms: our designed unmanned aerial vehicle combat (UAV-C) and StarCraft II Micromanagement (SCII-M). Simulation results demonstrate that the proposed MEARL can greatly improve the learning efficiency and performance of MASs and is superior to the state-of-the-art methods in multiagent tasks. |
源URL | [http://ir.ia.ac.cn/handle/173211/51872] |
专题 | 综合信息系统研究中心_飞行器智能技术 |
作者单位 | 1.中国电子科技集团 2.中国科学院大学人工智能学院 3.中国科学院自动化研究所 |
推荐引用方式 GB/T 7714 | Zhang TL,Liu Z,Yi JQ,et al. Multiexperience-Assisted Efficient Multiagent Reinforcement Learning[J]. IEEE Transactions on Neural Networks and Learning Systems,2023:1-15. |
APA | Zhang TL,Liu Z,Yi JQ,Wu SG,Pu ZQ,&Zhao YJ.(2023).Multiexperience-Assisted Efficient Multiagent Reinforcement Learning.IEEE Transactions on Neural Networks and Learning Systems,1-15. |
MLA | Zhang TL,et al."Multiexperience-Assisted Efficient Multiagent Reinforcement Learning".IEEE Transactions on Neural Networks and Learning Systems (2023):1-15. |
入库方式: OAI收割
来源:自动化研究所
其他版本
除非特别说明,本系统中所有内容都受版权保护,并保留所有权利。