Bird's-Eye-View Semantic Segmentation With Two-Stream Compact Depth Transformation and Feature Rectification
文献类型：期刊论文
| 作者 | Liu, Jierui3,4 ; Cao, Zhiqiang3,4; Yang, Jing3,4; Liu, Xilong3,4; Yang, Yuequan2; Qu, Zhiyou1
|
| 刊名 | IEEE TRANSACTIONS ON INTELLIGENT VEHICLES
 |
| 出版日期 | 2023-11-01 |
| 卷号 | 8期号:11页码:4546-4558 |
| 关键词 | Bird's-eye-view semantic segmentation two-stream compact depth transformation feature rectification |
| ISSN号 | 2379-8858 |
| DOI | 10.1109/TIV.2023.3275993 |
| 通讯作者 | Liu, Xilong(xilong.liu@ia.ac.cn) |
| 英文摘要 | Bird's-eye-view (BEV) perception has gained popularity since it provides a 3D world representation with scale consistency. Although existing camera-based solutions achieve excellent performance, the BEV positions related to features are still less accurate. In this article, a BEV semantic segmentation framework with two-stream compact depth transformation and feature rectification is proposed. To balance the conflict that the feature maps ensemble tends to use two temporal frames with long interval, while shorter temporal frames are more beneficial to depth prediction, a two-stream compact depth transformation is designed. Between original temporal frames, we introduce an intermediate frame to decouple the joint depth estimation of original frames. The local representations of the intermediate frame are respectively matched with each original temporal frame to achieve stereo depth predictions, where compact cost volumes are built to significantly reduce memory usage with high discriminability in depth-dimension. Further, virtual camera intrinsic parameters are derived to realize adaption of compact cost volume to various 2D data augmentation and improve generalization. On this basis, BEV feature maps are obtained via feature transformation. With the influence of depth distribution errors to BEV feature map, a feature rectified segmentation network is proposed to dynamically adjust the position offsets of input features via deformable convolution and semantic information-guided feature learning. As a result, a dense and accurate BEV semantic map is obtained. In addition, a self-supervised depth estimation teacher is adopted to provide extra supervision for depth prediction of our segmentation framework. The effectiveness of the proposed method is verified on public datasets. |
| WOS关键词 | NETWORK |
| 资助项目 | National Natural Science Foundation of China |
| WOS研究方向 | Computer Science ; Engineering ; Transportation |
| 语种 | 英语 |
| WOS记录号 | WOS:001128422200007 |
| 出版者 | IEEE-INST ELECTRICAL ELECTRONICS ENGINEERS INC |
| 资助机构 | National Natural Science Foundation of China |
| 源URL | [http://ir.ia.ac.cn/handle/173211/55667]  |
| 专题 | 多模态人工智能系统全国重点实验室 |
| 通讯作者 | Liu, Xilong |
| 作者单位 | 1.North Automat Control Technol Inst, Taiyuan 030006, Peoples R China 2.Yangzhou Univ, Coll Informat Engn, Artificial Intelligence Coll, Yangzhou 225009, Peoples R China 3.Univ Chinese Acad Sci, Sch Artificial Intelligence, Beijing 100049, Peoples R China 4.Chinese Acad Sci, Inst Automat, State Key Lab Multimodal Artificial Intelligence S, Beijing 100190, Peoples R China |
| 推荐引用方式 GB/T 7714 | Liu, Jierui,Cao, Zhiqiang,Yang, Jing,et al. Bird's-Eye-View Semantic Segmentation With Two-Stream Compact Depth Transformation and Feature Rectification[J]. IEEE TRANSACTIONS ON INTELLIGENT VEHICLES,2023,8(11):4546-4558. |
| APA | Liu, Jierui,Cao, Zhiqiang,Yang, Jing,Liu, Xilong,Yang, Yuequan,&Qu, Zhiyou.(2023).Bird's-Eye-View Semantic Segmentation With Two-Stream Compact Depth Transformation and Feature Rectification.IEEE TRANSACTIONS ON INTELLIGENT VEHICLES,8(11),4546-4558. |
| MLA | Liu, Jierui,et al."Bird's-Eye-View Semantic Segmentation With Two-Stream Compact Depth Transformation and Feature Rectification".IEEE TRANSACTIONS ON INTELLIGENT VEHICLES 8.11(2023):4546-4558. |
入库方式： OAI收割
来源：自动化研究所
浏览0
下载0
收藏0
其他版本
除非特别说明，本系统中所有内容都受版权保护，并保留所有权利。