Haisor: Human-aware Indoor Scene Optimization via Deep Reinforcement Learning
文献类型：期刊论文
| 作者 | Sun, Jia-Mu1; Yang, Jie1; Mo, Kaichun2; Lai, Yu-Kun3; Guibas, Leonidas2; Gao, Lin1 |
| 刊名 | ACM TRANSACTIONS ON GRAPHICS
 |
| 出版日期 | 2024-04-01 |
| 卷号 | 43期号:2页码:17 |
| 关键词 | Scene optimization scene synthesis human aware reinforcement learning Monte Carlo search robot simulation imitation learning |
| ISSN号 | 0730-0301 |
| DOI | 10.1145/3632947 |
| 英文摘要 | 3D scene synthesis facilitates and benefits many real-world applications. Most scene generators focus on making indoor scenes plausible via learning from training data and leveraging extra constraints such as adjacency and symmetry. Although the generated 3D scenes are mostly plausible with visually realistic layouts, they can be functionally unsuitable for human users to navigate and interact with furniture. Our key observation is that human activity plays a critical role and sufficient free space is essential for human-scene interactions. This is exactly where many existing synthesized scenes fail-the seemingly correct layouts are often not fit for living. To tackle this, we present a human-aware optimization framework Haisor for 3D indoor scene arrangement via reinforcement learning, which aims to find an action sequence to optimize the indoor scene layout automatically. Based on the hierarchical scene graph representation, an optimal action sequence is predicted and performed via Deep Q-Learning with Monte Carlo Tree Search (MCTS), where MCTS is our key feature to search for the optimal solution in long-term sequences and large action space. Multiple human-aware rewards are designed as our core criteria of human-scene interaction, aiming to identify the next smart action by leveraging powerful reinforcement learning. Our framework is optimized end-to-end by giving the indoor scenes with part-level furniture layout including part mobility information. Furthermore, our methodology is extensible and allows utilizing different reward designs to achieve personalized indoor scene synthesis. Extensive experiments demonstrate that our approach optimizes the layout of 3D indoor scenes in a human-aware manner, which is more realistic and plausible than original state-of-the-art generator results, and our approach produces superior smart actions, outperforming alternative baselines. |
| 资助项目 | National Natural Science Foundation of China[62322210] ; Beijing Municipal Natural Science Foundation for Distinguished Young Scholars[JQ21013] ; Beijing Municipal Science and Technology Commission[Z231100005923031] ; ARL[W911NF-21-2-0104] ; Vannevar Bush Faculty Fellowship |
| WOS研究方向 | Computer Science |
| 语种 | 英语 |
| WOS记录号 | WOS:001208809900002 |
| 出版者 | ASSOC COMPUTING MACHINERY |
| 源URL | [http://119.78.100.204/handle/2XEOYT63/39002]  |
| 专题 | 中国科学院计算技术研究所期刊论文_英文 |
| 通讯作者 | Gao, Lin |
| 作者单位 | 1.Chinese Acad Sci, Beijing Key Lab Mobile Comp & Pervas Device, Inst Comp Technol, Beijing 100190, Peoples R China 2.Stanford Univ, Dept Comp Sci, 450 Serra Mall, Stanford, CA 94305 USA 3.Cardiff Univ, Sch Comp Sci & Informat, Cardiff CF10 3AT, Wales |
| 推荐引用方式 GB/T 7714 | Sun, Jia-Mu,Yang, Jie,Mo, Kaichun,et al. Haisor: Human-aware Indoor Scene Optimization via Deep Reinforcement Learning[J]. ACM TRANSACTIONS ON GRAPHICS,2024,43(2):17. |
| APA | Sun, Jia-Mu,Yang, Jie,Mo, Kaichun,Lai, Yu-Kun,Guibas, Leonidas,&Gao, Lin.(2024).Haisor: Human-aware Indoor Scene Optimization via Deep Reinforcement Learning.ACM TRANSACTIONS ON GRAPHICS,43(2),17. |
| MLA | Sun, Jia-Mu,et al."Haisor: Human-aware Indoor Scene Optimization via Deep Reinforcement Learning".ACM TRANSACTIONS ON GRAPHICS 43.2(2024):17. |
入库方式： OAI收割
来源：计算技术研究所
浏览0
下载0
收藏0
其他版本
除非特别说明，本系统中所有内容都受版权保护，并保留所有权利。