Advancing Air Combat Tactics with Improved Neural Fictitious Self-Play Reinforcement Learning
文献类型：会议论文
| 作者 | He SQ(何少钦)1,2 ; Gao Y(高阳)1; Zhang BF(张保丰)1,2; Chang H(常惠)1; Zhang XC(张鑫辰)1
|
| 出版日期 | 2023-07 |
| 会议日期 | 2023-8 |
| 会议地点 | 中国郑州 |
| 关键词 | Air Combat, Reinforcement Learning, Neural Fictitious Self-Play. |
| 卷号 | 14090 |
| 期号 | Lecture Notes in Computer Science |
| DOI | https://doi.org/10.1007/978-981-99-4761-4_55 |
| 页码 | 653–666 |
| 英文摘要 | We study the problem of utilizing reinforcement learning for action
control in 1v1 Beyond-Visual-Range (BVR) air combat. In contrast to most
reinforcement learning problems, 1v1 BVR air combat belongs to the class of
two-player zero-sum games with long decision-making periods and sparse
rewards. The complexity of action and state space in this game makes it
difficult to learn high-level air combat strategies from scratch. To address this
problem, we propose a reinforcement learning self-play training framework to
solve it from two aspects: the decision model and the training algorithm. Our
decision-making model uses the Soft actor-critic (SAC) algorithm, a method
based on maximum entropy, as the action control of the reinforcement learning
part, and introduces an action mask to achieve efficient exploration. Our
training algorithm improves Neural Fictitious Self-Play (NFSP) and proposes
the best response history correction (BRHC) version of NFSP. These two
components helped our algorithm to achieve efficient training in the high
fidelity simulation environment. The result of the 1v1 BVR air combat problem
shows that the improved NFSP-BRHC algorithm outperforms both the NFSP
and the Self-Play (SP) algorithms. |
| 会议录 | Advanced Intelligent Computing Technology and Applications
 |
| 会议录出版者 | Springer |
| 会议录出版地 | Singapore |
| 语种 | 英语 |
| URL标识 | 查看原文 |
| 源URL | [http://ir.ia.ac.cn/handle/173211/57094]  |
| 专题 | 复杂系统认知与决策实验室 |
| 通讯作者 | Gao Y(高阳) |
| 作者单位 | 1.中国科学院自动化研究所 2.中国科学院大学 |
| 推荐引用方式 GB/T 7714 | He SQ,Gao Y,Zhang BF,et al. Advancing Air Combat Tactics with Improved Neural Fictitious Self-Play Reinforcement Learning[C]. 见:. 中国郑州. 2023-8. |
入库方式： OAI收割
来源：自动化研究所
浏览0
下载0
收藏0
其他版本
除非特别说明，本系统中所有内容都受版权保护，并保留所有权利。