Learning to Play Football From Sports Domain Perspective: A Knowledge-Embedded Deep Reinforcement Learning Framework
文献类型：期刊论文
| 作者 | Liu, Boyin1,2; Pu, Zhiqiang1,2 ; Zhang, Tianle1,2; Wang, Huimu3; Yi, Jianqiang1,2; Mi, Jiachen4
|
| 刊名 | IEEE TRANSACTIONS ON GAMES
 |
| 出版日期 | 2023-12-01 |
| 卷号 | 15期号:4页码:648-657 |
| ISSN号 | 2475-1502 |
| 关键词 | Deformable convolution football analysis pitch control reinforcement learning |
| DOI | 10.1109/TG.2022.3207068 |
| 通讯作者 | Pu, Zhiqiang(zhiqiang.pu@ia.ac.cn) |
| 英文摘要 | Applying deep reinforcement learning to football games has recently received extensive attention. However, this remains challenging due to the excessively high complexity of the football environment, such as high-dynamical game states, sparse rewards, and multiple roles with different capabilities. Existing works aim to address these problems without considering abundant domain knowledge of football. In this article, a football knowledge-embedded learning framework is proposed. Specifically, the pitch control concept is innovatively introduced to design a knowledge-embedded state representation. As a result, a novel pitch control model is designed that quantitatively provides space influence values of a single player, the whole team, and the ball. Different from existing models, this model additionally considers each player's various capabilities, including flexibility, explosive force, and stamina. Furthermore, the deformable convolution network is adopted for state representation extracting, which is used to process the geometric transformation of the players' positions and spatial influence values generated by the pitch control model. Then, based on this comprehensive state representation, a proximal policy optimization-based reinforcement learning scheme is adopted to generate the final policy. Finally, extensive simulations, including learning against a fixed opponent and learning from self-play, clearly show the effectiveness and adaptability of our proposed framework. |
| WOS关键词 | GAME |
| 资助项目 | National Key Research, Development Program of China |
| WOS研究方向 | Computer Science |
| 语种 | 英语 |
| 出版者 | IEEE-INST ELECTRICAL ELECTRONICS ENGINEERS INC |
| WOS记录号 | WOS:001128375200008 |
| 资助机构 | National Key Research, Development Program of China |
| 源URL | [http://ir.ia.ac.cn/handle/173211/54889]  |
| 专题 | 复杂系统认知与决策实验室 |
| 通讯作者 | Pu, Zhiqiang |
| 作者单位 | 1.Univ Chinese Acad Sci, Sch Artificial Intelligence, Beijing 100049, Peoples R China 2.Chinese Acad Sci, Inst Automat, Beijing 100190, Peoples R China 3.JD COM, Beijing 100176, Peoples R China 4.Beijing Technol & Business Univ, Sch Artificial Intelligence, Beijing 102401, Peoples R China |
| 推荐引用方式 GB/T 7714 | Liu, Boyin,Pu, Zhiqiang,Zhang, Tianle,et al. Learning to Play Football From Sports Domain Perspective: A Knowledge-Embedded Deep Reinforcement Learning Framework[J]. IEEE TRANSACTIONS ON GAMES,2023,15(4):648-657. |
| APA | Liu, Boyin,Pu, Zhiqiang,Zhang, Tianle,Wang, Huimu,Yi, Jianqiang,&Mi, Jiachen.(2023).Learning to Play Football From Sports Domain Perspective: A Knowledge-Embedded Deep Reinforcement Learning Framework.IEEE TRANSACTIONS ON GAMES,15(4),648-657. |
| MLA | Liu, Boyin,et al."Learning to Play Football From Sports Domain Perspective: A Knowledge-Embedded Deep Reinforcement Learning Framework".IEEE TRANSACTIONS ON GAMES 15.4(2023):648-657. |
入库方式： OAI收割
来源：自动化研究所
浏览0
下载0
收藏0
其他版本
除非特别说明，本系统中所有内容都受版权保护，并保留所有权利。