VPA: Multi-Modal Virtual Point Augmentation for 3D Object Detection
文献类型：期刊论文
| 作者 | Zhong, Jianping2; Qi, Zhaobo2; Duan, Kaiwen1,3; Xu, Yuanrong2; Zhang, Weigang2; Huang, Qingming1,3,4 |
| 刊名 | IEEE TRANSACTIONS ON CIRCUITS AND SYSTEMS FOR VIDEO TECHNOLOGY
 |
| 出版日期 | 2025-12-01 |
| 卷号 | 35期号:12页码:12410-12425 |
| 关键词 | Three-dimensional displays Point cloud compression Object detection Semantics Laser radar Feature extraction Detectors Accuracy Bicycles Solids 3D object detection multimodal fusion virtual point augmenting |
| ISSN号 | 1051-8215 |
| DOI | 10.1109/TCSVT.2025.3578474 |
| 英文摘要 | Integrating LiDAR and camera data is crucial for precise 3D object detection. Existing methods resort to augmenting virtual points from 2D image space in a random manner to complete the appearance of 3D objects with sparse points. However, these augmented virtual points have unreasonable 3D positions and representations, which brings serious negative effects on accurate detection. To this end, we introduce a general 3D object detection framework called Virtual Point Augmenting (VPA) to enrich the 3D point cloud by controllably generating virtual points with accurate depth and position information as well as domain-gap-eliminated multi-modal representations from image and point cloud spaces. VPA contains two core designs, namely Hybrid Sampling Method (HSM) and Fine-Grained Cross-modal Fusion (FGCF). HSM uses the constructed seed point distribution map based on the edge score and mask score map to sample high-quality seed points, and employs a feature similarity function to sample with k neighbors' depth to obtain more accurate depth for the seed points, thereby enhancing the quality of the virtual points' 3D positions. FGCF fuses the multi-modal features, i.e., the semantic feature, the geometric feature from the image space, and the 3D position feature in an adaptive manner using self-attention mechanism, thereby further improving the representation of the virtual points. We apply VPA to the LiDAR-based method CenterPoint and fusion-based method Cross-modal transformer. Experimental results on the nuScenes, KITTI, and Waymo benchmarks validate the efficiency of our VPA, which achieves promising performance with 72.9% mAP and 74.8% NDS without using test-time augmentation and model ensemble techniques on the nuScenes test set. Code is available at https://github.com/jianpingZhonggit/vpa.git |
| 资助项目 | National Natural Science Foundation of China[62441232] ; National Natural Science Foundation of China[62306092] ; National Natural Science Foundation of China[U21B2038] ; National Natural Science Foundation of China[62476068] ; National Natural Science Foundation of China[62236008] ; Natural Science Foundation of Shandong Province, China[ZR2024QF066] ; Natural Science Foundation of Shandong Province, China[ZR2023QF052] |
| WOS研究方向 | Engineering |
| 语种 | 英语 |
| WOS记录号 | WOS:001631874000019 |
| 出版者 | IEEE-INST ELECTRICAL ELECTRONICS ENGINEERS INC |
| 源URL | [http://119.78.100.204/handle/2XEOYT63/42985]  |
| 专题 | 中国科学院计算技术研究所 |
| 通讯作者 | Zhang, Weigang; Huang, Qingming |
| 作者单位 | 1.Univ Chinese Acad Sci, Key Lab Big Data Min & Knowledge Management, Beijing 100190, Peoples R China 2.Harbin Inst Technol, Sch Comp Sci & Technol, Weihai 264209, Peoples R China 3.Univ Chinese Acad Sci, Sch Comp Sci & Technol, Beijing 101408, Peoples R China 4.Chinese Acad Sci, Inst Comp Technol, Key Lab Intelligent Informat Proc, Beijing 100190, Peoples R China |
| 推荐引用方式 GB/T 7714 | Zhong, Jianping,Qi, Zhaobo,Duan, Kaiwen,et al. VPA: Multi-Modal Virtual Point Augmentation for 3D Object Detection[J]. IEEE TRANSACTIONS ON CIRCUITS AND SYSTEMS FOR VIDEO TECHNOLOGY,2025,35(12):12410-12425. |
| APA | Zhong, Jianping,Qi, Zhaobo,Duan, Kaiwen,Xu, Yuanrong,Zhang, Weigang,&Huang, Qingming.(2025).VPA: Multi-Modal Virtual Point Augmentation for 3D Object Detection.IEEE TRANSACTIONS ON CIRCUITS AND SYSTEMS FOR VIDEO TECHNOLOGY,35(12),12410-12425. |
| MLA | Zhong, Jianping,et al."VPA: Multi-Modal Virtual Point Augmentation for 3D Object Detection".IEEE TRANSACTIONS ON CIRCUITS AND SYSTEMS FOR VIDEO TECHNOLOGY 35.12(2025):12410-12425. |
入库方式： OAI收割
来源：计算技术研究所
浏览0
下载0
收藏0
其他版本
除非特别说明，本系统中所有内容都受版权保护，并保留所有权利。