ACF-Net: Asymmetric Cascade Fusion for 3D Detection With LiDAR Point Clouds and Images
文献类型：期刊论文
| 作者 | Tian, Yonglin1,2 ; Zhang, Xianjing2; Wang, Xiao3; Xu, Jintao2; Wang, Jiangong1; Ai, Rui4; Gu, Weihao4; Ding, Weiping5
|
| 刊名 | IEEE TRANSACTIONS ON INTELLIGENT VEHICLES
 |
| 出版日期 | 2024-02-01 |
| 卷号 | 9期号:2页码:3360-3371 |
| 关键词 | Three-dimensional displays Feature extraction Point cloud compression Laser radar Object detection Timing Fuses 3D detection autonomous driving asymmetric fusion cascade fusion multimodal fusion |
| ISSN号 | 2379-8858 |
| DOI | 10.1109/TIV.2023.3341223 |
| 通讯作者 | Wang, Xiao(xiao.wang@ahu.edu.cn) |
| 英文摘要 | The recognition and utilization of complementary information arising from modality-intrinsic properties play crucial roles in multimodal 3D detection. However, most of the current approaches for fusion-based 3D detection follow symmetrical fusion paradigms and adopt early fusion, middle fusion as well as late fusion styles, which ignore the unequal status of data with different modalities. In this paper, according to the timing of fusion, we adopt an asymmetric cascade fusion network to exploit both the structural information from point clouds and the complementary semantic information from images. A multi-stage cascade design of 3D object detection is proposed to iteratively refine predictions and several late image features (comprised of detection clues, segmentation clues, and deep features from encoders) are incorporated into different stages of the LiDAR branch to maintain the integrity of image features and enable deep multimodal interactions. Besides, to mitigate the effects of the down-sampling of voxelized features and possible mismatching of multimodal data, we propose proxy-based cross-modality sampling to utilize the high-density point clouds coordinates and develop an image degeneration process to simulate the noise in cross-modality matching for robust training. Extensive experiments are conducted on KITTI and Waymo Open Dataset, which validate the effectiveness of the proposed method. |
| WOS关键词 | OBJECT ; PERFORMANCE |
| 资助项目 | Key-Area Research and Development Program of Guangdong Province |
| WOS研究方向 | Computer Science ; Engineering ; Transportation |
| 语种 | 英语 |
| WOS记录号 | WOS:001215322100045 |
| 出版者 | IEEE-INST ELECTRICAL ELECTRONICS ENGINEERS INC |
| 资助机构 | Key-Area Research and Development Program of Guangdong Province |
| 源URL | [http://ir.ia.ac.cn/handle/173211/59064]  |
| 专题 | 自动化研究所_复杂系统管理与控制国家重点实验室_先进控制与自动化团队 |
| 通讯作者 | Wang, Xiao |
| 作者单位 | 1.Chinese Acad Sci, Inst Automat, State Key Lab Management & Control Complex Syst, Beijing 100190, Peoples R China 2.Haomo Technol Co Ltd, AI Ctr, Beijing 100192, Peoples R China 3.Anhui Univ, Sch Artificial Intelligence, Hefei 230031, Peoples R China 4.Haomo Technol Co Ltd, Beijing 100192, Peoples R China 5.Nantong Univ, Sch Informat Sci & Technol, Nantong 226019, Peoples R China |
| 推荐引用方式 GB/T 7714 | Tian, Yonglin,Zhang, Xianjing,Wang, Xiao,et al. ACF-Net: Asymmetric Cascade Fusion for 3D Detection With LiDAR Point Clouds and Images[J]. IEEE TRANSACTIONS ON INTELLIGENT VEHICLES,2024,9(2):3360-3371. |
| APA | Tian, Yonglin.,Zhang, Xianjing.,Wang, Xiao.,Xu, Jintao.,Wang, Jiangong.,...&Ding, Weiping.(2024).ACF-Net: Asymmetric Cascade Fusion for 3D Detection With LiDAR Point Clouds and Images.IEEE TRANSACTIONS ON INTELLIGENT VEHICLES,9(2),3360-3371. |
| MLA | Tian, Yonglin,et al."ACF-Net: Asymmetric Cascade Fusion for 3D Detection With LiDAR Point Clouds and Images".IEEE TRANSACTIONS ON INTELLIGENT VEHICLES 9.2(2024):3360-3371. |
入库方式： OAI收割
来源：自动化研究所
浏览0
下载0
收藏0
其他版本
除非特别说明，本系统中所有内容都受版权保护，并保留所有权利。