Discovering Latent Variables for the Tasks With Confounders in Multi-Agent Reinforcement Learning
文献类型：期刊论文
| 作者 | Kun Jiang; Wenzhang Liu; Yuanda Wang; Lu Dong; Changyin Sun |
| 刊名 | IEEE/CAA Journal of Automatica Sinica
 |
| 出版日期 | 2024 |
| 卷号 | 11期号:7页码:1591-1604 |
| 关键词 | Latent variable model maximum entropy multi-agent reinforcement learning (MARL) multi-agent system |
| ISSN号 | 2329-9266 |
| DOI | 10.1109/JAS.2024.124281 |
| 英文摘要 | Efficient exploration in complex coordination tasks has been considered a challenging problem in multi-agent reinforcement learning (MARL). It is significantly more difficult for those tasks with latent variables that agents cannot directly observe. However, most of the existing latent variable discovery methods lack a clear representation of latent variables and an effective evaluation of the influence of latent variables on the agent. In this paper, we propose a new MARL algorithm based on the soft actor-critic method for complex continuous control tasks with confounders. It is called the multi-agent soft actor-critic with latent variable (MASAC-LV) algorithm, which uses variational inference theory to infer the compact latent variables representation space from a large amount of offline experience. Besides, we derive the counterfactual policy whose input has no latent variables and quantify the difference between the actual policy and the counterfactual policy via a distance function. This quantified difference is considered an intrinsic motivation that gives additional rewards based on how much the latent variable affects each agent. The proposed algorithm is evaluated on two collaboration tasks with confounders, and the experimental results demonstrate the effectiveness of MASAC-LV compared to other baseline algorithms. |
| 源URL | [http://ir.ia.ac.cn/handle/173211/57304]  |
| 专题 | 自动化研究所_学术期刊_IEEE/CAA Journal of Automatica Sinica |
| 推荐引用方式 GB/T 7714 | Kun Jiang,Wenzhang Liu,Yuanda Wang,et al. Discovering Latent Variables for the Tasks With Confounders in Multi-Agent Reinforcement Learning[J]. IEEE/CAA Journal of Automatica Sinica,2024,11(7):1591-1604. |
| APA | Kun Jiang,Wenzhang Liu,Yuanda Wang,Lu Dong,&Changyin Sun.(2024).Discovering Latent Variables for the Tasks With Confounders in Multi-Agent Reinforcement Learning.IEEE/CAA Journal of Automatica Sinica,11(7),1591-1604. |
| MLA | Kun Jiang,et al."Discovering Latent Variables for the Tasks With Confounders in Multi-Agent Reinforcement Learning".IEEE/CAA Journal of Automatica Sinica 11.7(2024):1591-1604. |
入库方式： OAI收割
来源：自动化研究所
浏览0
下载0
收藏0
其他版本
除非特别说明，本系统中所有内容都受版权保护，并保留所有权利。