Subspace-Aware Exploration for Sparse-Reward Multi-Agent Tasks
文献类型:会议论文
作者 | Pei Xu2,3; Junge Zhang2; Qiyue Yin2; Chao Yu4; Yaodong Yang1,5; Kaiqi Huang2,3,6 |
出版日期 | 2023-02-14 |
会议日期 | 2023-2-7 |
会议地点 | Washington DC, USA |
关键词 | deep reinforcement learning sparse reward exploration multi-agent |
英文摘要 | Exploration under sparse rewards is a key challenge for multi agent reinforcement learning problems. One possible solution to this issue is to exploit inherent task structures for an acceleration of exploration. In this paper, we present a novel exploration approach, which encodes a special structural prior on the reward function into exploration, for sparse-reward multi agent tasks. Specifically, a novel entropic exploration objective which encodes the structural prior is proposed to accelerate the discovery of rewards. By maximizing the lower bound of this objective, we then propose an algorithm with moderate computational cost, which can be applied to practical tasks. Under the sparse-reward setting, we show that the proposed algorithm significantly outperforms the state-of-the-art algorithms in the multiple-particle environment, the Google Research Football and StarCraft II micromanagement tasks. To the best of our knowledge, on some hard tasks (such as 27m vs 30m) which have relatively larger number of agents and need non-trivial strategies to defeat enemies, our method is the first to learn winning strategies under the sparse-reward setting. |
会议录出版者 | Association for the Advancement of Artificial Intelligence |
语种 | 英语 |
源URL | [http://ir.ia.ac.cn/handle/173211/52050] |
专题 | 智能系统与工程 |
作者单位 | 1.Beijing Institute for General AI 2.CRISE, Institute of Automation, Chinese Academy of Sciences 3.School of Artificial Intelligence, University of Chinese Academy of Sciences 4.School of Computer Science and Engineering, Sun Yat-sen University 5.Institute for AI, Peking University 6.CAS, Center for Excellence in Brain Science and Intelligence Technology |
推荐引用方式 GB/T 7714 | Pei Xu,Junge Zhang,Qiyue Yin,et al. Subspace-Aware Exploration for Sparse-Reward Multi-Agent Tasks[C]. 见:. Washington DC, USA. 2023-2-7. |
入库方式: OAI收割
来源:自动化研究所
其他版本
除非特别说明,本系统中所有内容都受版权保护,并保留所有权利。