Data-based approximate policy iteration for affine nonlinear continuous-time optimal control design
文献类型：期刊论文
| 作者 | Luo, Biao1 ; Wu, Huai-Ning2; Huang, Tingwen3; Liu, Derong1
|
| 刊名 | AUTOMATICA
 |
| 出版日期 | 2014-12-01 |
| 卷号 | 50期号:12页码:3281-3290 |
| 关键词 | Nonlinear optimal control Reinforcement learning Off-policy Data-based approximate policy iteration Neural network Hamilton-Jacobi-Bellman equation |
| 英文摘要 | This paper addresses the model-free nonlinear optimal control problem based on data by introducing the reinforcement learning (RL) technique. It is known that the nonlinear optimal control problem relies on the solution of the Hamilton-Jacobi-Bellman (HJB) equation, which is a nonlinear partial differential equation that is generally impossible to be solved analytically. Even worse, most practical systems are too complicated to establish an accurate mathematical model. To overcome these difficulties, we propose a data-based approximate policy iteration (API) method by using real system data rather than a system model. Firstly, a model-free policy iteration algorithm is derived and its convergence is proved. The implementation of the algorithm is based on the actor-critic structure, where actor and critic neural networks (NNs) are employed to approximate the control policy and cost function, respectively. To update the weights of actor and critic NNs, a least-square approach is developed based on the method of weighted residuals. The data-based API is an off-policy RL method, where the "exploration" is improved by arbitrarily sampling data on the state and input domain. Finally, we test the data-based API control design method on a simple nonlinear system, and further apply it to a rotational/translational actuator system. The simulation results demonstrate the effectiveness of the proposed method. (C) 2014 Elsevier Ltd. All rights reserved. |
| WOS标题词 | Science & Technology ; Technology |
| 类目[WOS] | Automation & Control Systems ; Engineering, Electrical & Electronic |
| 研究领域[WOS] | Automation & Control Systems ; Engineering |
| 关键词[WOS] | ADAPTIVE OPTIMAL-CONTROL ; LINEAR-SYSTEMS ; REINFORCEMENT ; STABILIZATION ; EQUATION |
| 收录类别 | SCI |
| 语种 | 英语 |
| WOS记录号 | WOS:000347760100036 |
| 源URL | [http://ir.ia.ac.cn/handle/173211/3824]  |
| 专题 | 自动化研究所_复杂系统管理与控制国家重点实验室_智能化团队 |
| 作者单位 | 1.Chinese Acad Sci, Inst Automat, State Key Lab Management & Control Complex Syst, Beijing 100190, Peoples R China 2.Beijing Univ Aeronaut & Astronaut, Beihang Univ, Sci & Technol Aircraft Control Lab, Beijing 100191, Peoples R China 3.Texas A&M Univ Qatar, Doha, Qatar |
| 推荐引用方式 GB/T 7714 | Luo, Biao,Wu, Huai-Ning,Huang, Tingwen,et al. Data-based approximate policy iteration for affine nonlinear continuous-time optimal control design[J]. AUTOMATICA,2014,50(12):3281-3290. |
| APA | Luo, Biao,Wu, Huai-Ning,Huang, Tingwen,&Liu, Derong.(2014).Data-based approximate policy iteration for affine nonlinear continuous-time optimal control design.AUTOMATICA,50(12),3281-3290. |
| MLA | Luo, Biao,et al."Data-based approximate policy iteration for affine nonlinear continuous-time optimal control design".AUTOMATICA 50.12(2014):3281-3290. |
入库方式： OAI收割
来源：自动化研究所
浏览0
下载0
收藏0
其他版本
除非特别说明，本系统中所有内容都受版权保护，并保留所有权利。