Instance-Aware Monocular 3D Semantic Scene Completion
文献类型：期刊论文
| 作者 | Xiao, Haihong2; Xu, Hongbin2; Kang, Wenxiong2; Li, Yuqiong1 ; Li YQ(李玉琼)
|
| 刊名 | IEEE TRANSACTIONS ON INTELLIGENT TRANSPORTATION SYSTEMS
 |
| 出版日期 | 2024-01-02 |
| 页码 | 12 |
| 关键词 | 3D scene understanding semantic scene completion 3D vision |
| ISSN号 | 1524-9050 |
| DOI | 10.1109/TITS.2023.3344806 |
| 通讯作者 | Kang, Wenxiong(auwxkang@scut.edu.cn) |
| 英文摘要 | We study outdoor 3D scene understanding, a challenging task demanding the intelligent system to infer both geometry and semantics from a single-view image - a critical skill for autonomous vehicles to navigate in the real 3D world. Towards this end, we present an instance-aware monocular semantic scene completion framework. To the best of our knowledge, this is the first endeavor specifically targeting the challenge of instance perception in the camera-based semantic scene completion task. Our method consists of two stages. In stage I, we design a region-based VQ-VAE network, providing an effective solution for 3D occupancy prediction. In stage II, we first introduce an instance-aware attention module, explicitly incorporating instance-level cues captured from mask images to enhance the instance features in RGB images. Then we leverage the deformable cross-attention to aggregate image features corresponding to each voxel query and utilize the deformable self-attention to refine query proposals. We combine these key ingredients and evaluate our method on two challenging datasets, namely SemanticKITTI and SSCBench-KITTI-360. The results unequivocally demonstrate the superiority of our proposed method over the state-of-the-art VoxFormer-S. Specifically, our method surpasses VoxFormer-S by 0.22 IoU and 0.72 mIoU on the validation set and achieves an impressive improvement of 3.04 IoU and 1.06 mIoU on the SSCBench-KITTI-360 validation set. Meanwhile, our approach ensures accurate perception of critical instances, thereby exhibiting its exceptional performance and potential for practical deployment. |
| 资助项目 | National Natural Science Foundation of China |
| WOS研究方向 | Engineering ; Transportation |
| 语种 | 英语 |
| WOS记录号 | WOS:001167317900001 |
| 资助机构 | National Natural Science Foundation of China |
| 源URL | [http://dspace.imech.ac.cn/handle/311007/94549]  |
| 专题 | 力学研究所_流固耦合系统力学重点实验室(2012-) |
| 通讯作者 | Kang, Wenxiong |
| 作者单位 | 1.Chinese Acad Sci, Inst Mech, Key Lab Mech Fluid Solid Coupling Syst, Beijing 100190, Peoples R China 2.South China Univ Technol, Sch Automat Sci & Engn, Guangzhou 511442, Peoples R China |
| 推荐引用方式 GB/T 7714 | Xiao, Haihong,Xu, Hongbin,Kang, Wenxiong,et al. Instance-Aware Monocular 3D Semantic Scene Completion[J]. IEEE TRANSACTIONS ON INTELLIGENT TRANSPORTATION SYSTEMS,2024:12. |
| APA | Xiao, Haihong,Xu, Hongbin,Kang, Wenxiong,Li, Yuqiong,&李玉琼.(2024).Instance-Aware Monocular 3D Semantic Scene Completion.IEEE TRANSACTIONS ON INTELLIGENT TRANSPORTATION SYSTEMS,12. |
| MLA | Xiao, Haihong,et al."Instance-Aware Monocular 3D Semantic Scene Completion".IEEE TRANSACTIONS ON INTELLIGENT TRANSPORTATION SYSTEMS (2024):12. |
入库方式： OAI收割
来源：力学研究所
浏览0
下载0
收藏0
其他版本
除非特别说明，本系统中所有内容都受版权保护，并保留所有权利。