Deep Reinforcement Learning With Part-Aware Exploration Bonus in Video Games
文献类型：期刊论文
| 作者 | Xu, Pei4,5 ; Yin, Qiyue3,4; Zhang, Junge3,4; Huang, Kaiqi1,2,3,4
|
| 刊名 | IEEE TRANSACTIONS ON GAMES
 |
| 出版日期 | 2022-12-01 |
| 卷号 | 14期号:4页码:644-653 |
| ISSN号 | 2475-1502 |
| 关键词 | Deep learning exploration reinforcement learning video game |
| DOI | 10.1109/TG.2021.3134259 |
| 通讯作者 | Zhang, Junge(jgzhang@nlpr.ia.ac.cn) |
| 英文摘要 | Reinforcement learning algorithms rely on carefully engineering environment rewards that are extrinsic to agents. However, environments with dense rewards are rare, motivating the need for developing reward functions that are intrinsic to agents. Curiosity is a type of successful intrinsic reward function, which uses the prediction error as an reward signal. In prior work, the prediction problem used to generate intrinsic rewards is optimized in the pixel space rather than a learnable feature space to avoid randomness caused by feature changes. However, these methods ignore small but important elements of the states that are often associated with locations of the character, which makes it impossible to generate accurate internal rewards for efficient exploration. In this article, we first demonstrate the effectiveness of introducing prior learned features for existing prediction-based exploration methods. Then, an attention map mechanism is designed to discretize learned features, thereby updating the learned feature and meanwhile reducing the impact of randomness on intrinsic rewards caused by the learning process of features. We verify our method on some video games from the standard reinforcement learning Atari benchmark, achieving clear improvements over random network distillation, which is one of the most advanced exploration methods, in almost all Atari games. |
| 资助项目 | National Natural Science Foundation of China[61876181] ; National Natural Science Foundation of China[61721004] ; Beijing Nova Program of Science and Technology[Z191100001119043] ; Youth Innovation Promotion Association, Chinese Academy of Sciences |
| WOS研究方向 | Computer Science |
| 语种 | 英语 |
| 出版者 | IEEE-INST ELECTRICAL ELECTRONICS ENGINEERS INC |
| WOS记录号 | WOS:000908822400011 |
| 资助机构 | National Natural Science Foundation of China ; Beijing Nova Program of Science and Technology ; Youth Innovation Promotion Association, Chinese Academy of Sciences |
| 源URL | [http://ir.ia.ac.cn/handle/173211/51051]  |
| 专题 | 智能系统与工程 |
| 通讯作者 | Zhang, Junge |
| 作者单位 | 1.CAS Ctr Excellence Brain Sci & Intelligence Techno, Beijing 100190, Peoples R China 2.Chinese Acad Sci, Natl Lab Pattern Recognit, Inst Automat, Beijing 100190, Peoples R China 3.Univ Chinese Acad Sci, Beijing 100049, Peoples R China 4.Chinese Acad Sci, Inst Automat, Ctr Res Intelligent Syst & Engn, Beijing 100190, Peoples R China 5.Univ Chinese Acad Sci, Sch Artificial Intelligence, Beijing 100049, Peoples R China |
| 推荐引用方式 GB/T 7714 | Xu, Pei,Yin, Qiyue,Zhang, Junge,et al. Deep Reinforcement Learning With Part-Aware Exploration Bonus in Video Games[J]. IEEE TRANSACTIONS ON GAMES,2022,14(4):644-653. |
| APA | Xu, Pei,Yin, Qiyue,Zhang, Junge,&Huang, Kaiqi.(2022).Deep Reinforcement Learning With Part-Aware Exploration Bonus in Video Games.IEEE TRANSACTIONS ON GAMES,14(4),644-653. |
| MLA | Xu, Pei,et al."Deep Reinforcement Learning With Part-Aware Exploration Bonus in Video Games".IEEE TRANSACTIONS ON GAMES 14.4(2022):644-653. |
入库方式： OAI收割
来源：自动化研究所
浏览0
下载0
收藏0
其他版本
除非特别说明，本系统中所有内容都受版权保护，并保留所有权利。