中国科学院机构知识库网格
Chinese Academy of Sciences Institutional Repositories Grid
首页 机构 成果 学者
  
  1. CAS IR Grid
  2. 自动化研究所
  3. 中国科学院自动化研究所
  4. 多模态人工智能系统全国重点实验室
  5. 深度强化学习
UNMAS: Multiagent Reinforcement Learning for Unshaped Cooperative Scenarios

文献类型：期刊论文

作者Chai, Jiajun1,3; Li, Weifan1,3; Zhu, Yuanheng1,3; Zhao, Dongbin1,3; Ma, Zhe2; Sun, Kewu2; Ding, Jishiyu2
刊名IEEE TRANSACTIONS ON NEURAL NETWORKS AND LEARNING SYSTEMS
出版日期2021-08-27
页码12
关键词Multi-agent systems Training Task analysis Reinforcement learning Sun Learning systems Semantics Centralized training with decentralized execution (CTDE) multiagent reinforcement learning StarCraft II
ISSN号2162-237X
DOI10.1109/TNNLS.2021.3105869
通讯作者Zhao, Dongbin(dongbin.zhao@ia.ac.cn)
英文摘要Multiagent reinforcement learning methods, such as VDN, QMIX, and QTRAN, that adopt centralized training with decentralized execution (CTDE) framework have shown promising results in cooperation and competition. However, in some multiagent scenarios, the number of agents and the size of the action set actually vary over time. We call these unshaped scenarios, and the methods mentioned above fail in performing satisfyingly. In this article, we propose a new method, called Unshaped Networks for Multiagent Systems (UNMAS), which adapts to the number and size changes in multiagent systems. We propose the self-weighting mixing network to factorize the joint action-value. Its adaption to the change in agent number is attributed to the nonlinear mapping from each-agent Q value to the joint action-value with individual weights. Besides, in order to address the change in an action set, each agent constructs an individual action-value network that is composed of two streams to evaluate the constant environment-oriented subset and the varying unit-oriented subset. We evaluate UNMAS on various StarCraft II micromanagement scenarios and compare the results with several state-of-the-art MARL algorithms. The superiority of UNMAS is demonstrated by its highest winning rates especially on the most difficult scenario 3s5z_vs_3s6z. The agents learn to perform effectively cooperative behaviors, while other MARL algorithms fail. Animated demonstrations and source code are provided in https://sites.google.com/view/unmas.
资助项目National Key Research and Development Program of China[2018AAA0102404] ; Strategic Priority Research Program of Chinese Academy of Sciences (CAS)[XDA27030400] ; Youth Innovation Promotion Association of CAS
WOS研究方向Computer Science ; Engineering
语种英语
WOS记录号WOS:000733450200001
出版者IEEE-INST ELECTRICAL ELECTRONICS ENGINEERS INC
资助机构National Key Research and Development Program of China ; Strategic Priority Research Program of Chinese Academy of Sciences (CAS) ; Youth Innovation Promotion Association of CAS
源URL[http://ir.ia.ac.cn/handle/173211/46989]  
专题复杂系统管理与控制国家重点实验室_深度强化学习
通讯作者Zhao, Dongbin
作者单位1.Univ Chinese Acad Sci, Sch Artificial Intelligence, Beijing 100049, Peoples R China
2.Second Acad CASIS, X Lab, Beijing 100854, Peoples R China
3.Chinese Acad Sci, Inst Automat, State Key Lab Management & Control Complex Syst, Beijing 100190, Peoples R China
推荐引用方式
GB/T 7714		 Chai, Jiajun,Li, Weifan,Zhu, Yuanheng,et al. UNMAS: Multiagent Reinforcement Learning for Unshaped Cooperative Scenarios[J]. IEEE TRANSACTIONS ON NEURAL NETWORKS AND LEARNING SYSTEMS,2021:12.
APA Chai, Jiajun.,Li, Weifan.,Zhu, Yuanheng.,Zhao, Dongbin.,Ma, Zhe.,...&Ding, Jishiyu.(2021).UNMAS: Multiagent Reinforcement Learning for Unshaped Cooperative Scenarios.IEEE TRANSACTIONS ON NEURAL NETWORKS AND LEARNING SYSTEMS,12.
MLA Chai, Jiajun,et al."UNMAS: Multiagent Reinforcement Learning for Unshaped Cooperative Scenarios".IEEE TRANSACTIONS ON NEURAL NETWORKS AND LEARNING SYSTEMS (2021):12.

入库方式： OAI收割

来源：自动化研究所

浏览0
下载0
收藏0
其他版本

除非特别说明，本系统中所有内容都受版权保护，并保留所有权利。