An Optimal Control-Based Distributed Reinforcement Learning Framework for A Class of Non-Convex Objective Functionals of the Multi-Agent Network
文献类型：期刊论文
| 作者 | Zhe Chen; Ning Li |
| 刊名 | IEEE/CAA Journal of Automatica Sinica
 |
| 出版日期 | 2023 |
| 卷号 | 10期号:11页码:2081-2093 |
| 关键词 | Distributed optimization multi-agent optimal control reinforcement learning (RL) |
| ISSN号 | 2329-9266 |
| DOI | 10.1109/JAS.2022.105992 |
| 英文摘要 | This paper studies a novel distributed optimization problem that aims to minimize the sum of the non-convex objective functionals of the multi-agent network under privacy protection, which means that the local objective of each agent is unknown to others. The above problem involves complexity simultaneously in the time and space aspects. Yet existing works about distributed optimization mainly consider privacy protection in the space aspect where the decision variable is a vector with finite dimensions. In contrast, when the time aspect is considered in this paper, the decision variable is a continuous function concerning time. Hence, the minimization of the overall functional belongs to the calculus of variations. Traditional works usually aim to seek the optimal decision function. Due to privacy protection and non-convexity, the Euler-Lagrange equation of the proposed problem is a complicated partial differential equation. Hence, we seek the optimal decision derivative function rather than the decision function. This manner can be regarded as seeking the control input for an optimal control problem, for which we propose a centralized reinforcement learning (RL) framework. In the space aspect, we further present a distributed reinforcement learning framework to deal with the impact of privacy protection. Finally, rigorous theoretical analysis and simulation validate the effectiveness of our framework. |
| 源URL | [http://ir.ia.ac.cn/handle/173211/52425]  |
| 专题 | 自动化研究所_学术期刊_IEEE/CAA Journal of Automatica Sinica |
| 推荐引用方式 GB/T 7714 | Zhe Chen,Ning Li. An Optimal Control-Based Distributed Reinforcement Learning Framework for A Class of Non-Convex Objective Functionals of the Multi-Agent Network[J]. IEEE/CAA Journal of Automatica Sinica,2023,10(11):2081-2093. |
| APA | Zhe Chen,&Ning Li.(2023).An Optimal Control-Based Distributed Reinforcement Learning Framework for A Class of Non-Convex Objective Functionals of the Multi-Agent Network.IEEE/CAA Journal of Automatica Sinica,10(11),2081-2093. |
| MLA | Zhe Chen,et al."An Optimal Control-Based Distributed Reinforcement Learning Framework for A Class of Non-Convex Objective Functionals of the Multi-Agent Network".IEEE/CAA Journal of Automatica Sinica 10.11(2023):2081-2093. |
入库方式： OAI收割
来源：自动化研究所
浏览0
下载0
收藏0
其他版本
除非特别说明，本系统中所有内容都受版权保护，并保留所有权利。