Pseudo Value Network Distillation for High-Performance Exploration
文献类型:会议论文
作者 | Zhao EM(赵恩民)1,3; Xing JL(兴军亮)2; Li K(李凯)3; Kang YX(康永欣)1,3; Tao P(陶品)2 |
出版日期 | 2023-04 |
会议日期 | 2023-06 |
会议地点 | 澳大利亚 |
DOI | 无 |
英文摘要 | Solving hard exploration tasks with sparse rewards is notoriously challenging in reinforcement learning (RL), which needs to address two key issues simultaneously: exploiting past successful experiences and exploring the unknown environment. Many prior works take expert demonstrations as successful experiences and learn to imitate them directly. However, these demonstrations are often not available in practice. Recently, curiosity-driven RL methods provide intrinsic rewards, encouraging the agent to explore states with high novelty. Nonetheless, they lack a mechanism for leveraging past good experiences effectively. This work presents a Pseudo Value Network Distillation (PVND) framework to balance the RL agent's exploitative and exploratory behaviors effectively and automatically. In particular, PVND learns to set high exploitation bonuses to the critical states in rewarded trajectories from past experiences and high exploration bonuses to the novel states that agents rarely visit during exploration. We theoretically demonstrate that PVND gives larger positive intrinsic rewards to more critical states. Furthermore, PVND automatically finds meaningful and critical hierarchical sub-tasks for agents to accomplish the final goal progressively. Competitive results in several hard exploration sparse reward problems have verified its effectiveness and efficiency. |
语种 | 英语 |
URL标识 | 查看原文 |
源URL | [http://ir.ia.ac.cn/handle/173211/52243] |
专题 | 融合创新中心_决策指挥与体系智能 |
通讯作者 | Xing JL(兴军亮) |
作者单位 | 1.School of Artificial Intelligence, University of Chinese Academy of Sciences 2.Department of Computer Science and Technology, Tsinghua University 3.Institute of Automation, Chinese Academy of Sciences |
推荐引用方式 GB/T 7714 | Zhao EM,Xing JL,Li K,et al. Pseudo Value Network Distillation for High-Performance Exploration[C]. 见:. 澳大利亚. 2023-06. |
入库方式: OAI收割
来源:自动化研究所
其他版本
除非特别说明,本系统中所有内容都受版权保护,并保留所有权利。