Collaborative Deconvolutional Neural Networks for Joint Depth Estimation and Semantic Segmentation
文献类型：期刊论文
| 作者 | Liu, Jing1 ; Wang, Yuhang1,2; Li, Yong1,2; Fu, Jun1,2; Li, Jiangyun3; Lu, Hanqing1
|
| 刊名 | IEEE TRANSACTIONS ON NEURAL NETWORKS AND LEARNING SYSTEMS
 |
| 出版日期 | 2018-11-01 |
| 卷号 | 29期号:11页码:5655-5666 |
| 关键词 | Deconvolutional neural network (DCNN) depth estimation fully connected conditional random field (CRF) pointwise bilinear layer semantic segmentation soft mapping strategy |
| ISSN号 | 2162-237X |
| DOI | 10.1109/TNNLS.2017.2787781 |
| 通讯作者 | Liu, Jing(jliu@nlpr.ia.ac.cn) |
| 英文摘要 | Semantic segmentation and single-view depth estimation are two fundamental problems in computer vision. They exploit the semantic and geometric properties of images, respectively, and are thus complementary in scene understanding. In this paper, we propose a collaborative deconvolutional neural network (C-DCNN) to jointly model these two problems for mutual promotion. The C-DCNN consists of two DCNNs, of which each is for one task. The DCNNs provide a finer resolution reconstruction method and are pretrained with hierarchical supervision. The feature maps from these two DCNNs are integrated via a pointwise bilinear layer, which fuses the semantic and depth information and produces higher order features. Then, the integrated features are fed into two sibling classification layers to simultaneously learn for semantic segmentation and depth estimation. In this way, we combine the semantic and depth features in a unified deep network and jointly train them to benefit each other. Specifically, during network training, we process depth estimation as a classification problem where a soft mapping strategy is proposed to map the continuous depth values into discrete probability distributions and the cross entropy loss is used. Besides, a fully connected conditional random field is also used as postprocessing to further improve the performance of semantic segmentation, where the proximity relations of pixels on position, intensity, and depth are jointly considered. We evaluate our approach on two challenging benchmarks: NYU Depth V2 and SUN RGB-D. It is demonstrated that our approach effectively utilizes these two kinds of information and achieves state-of-the-art results on both the semantic segmentation and depth estimation tasks. |
| WOS关键词 | RECOGNITION |
| 资助项目 | National Natural Science Foundation of China[61332016] ; National Natural Science Foundation of China[61472422] ; Fundamental Research Funds for Central Universities[FRFBD-16-005A] |
| WOS研究方向 | Computer Science ; Engineering |
| 语种 | 英语 |
| WOS记录号 | WOS:000447832200038 |
| 出版者 | IEEE-INST ELECTRICAL ELECTRONICS ENGINEERS INC |
| 资助机构 | National Natural Science Foundation of China ; Fundamental Research Funds for Central Universities |
| 源URL | [http://ir.ia.ac.cn/handle/173211/28101]  |
| 专题 | 自动化研究所_模式识别国家重点实验室_图像与视频分析团队 |
| 通讯作者 | Liu, Jing |
| 作者单位 | 1.Chinese Acad Sci, Inst Automat, Natl Lab Pattern Recognit, Beijing 100190, Peoples R China 2.Univ Chinese Acad Sci, Beijing 100049, Peoples R China 3.Univ Sci & Technol, Beijing 100083, Peoples R China |
| 推荐引用方式 GB/T 7714 | Liu, Jing,Wang, Yuhang,Li, Yong,et al. Collaborative Deconvolutional Neural Networks for Joint Depth Estimation and Semantic Segmentation[J]. IEEE TRANSACTIONS ON NEURAL NETWORKS AND LEARNING SYSTEMS,2018,29(11):5655-5666. |
| APA | Liu, Jing,Wang, Yuhang,Li, Yong,Fu, Jun,Li, Jiangyun,&Lu, Hanqing.(2018).Collaborative Deconvolutional Neural Networks for Joint Depth Estimation and Semantic Segmentation.IEEE TRANSACTIONS ON NEURAL NETWORKS AND LEARNING SYSTEMS,29(11),5655-5666. |
| MLA | Liu, Jing,et al."Collaborative Deconvolutional Neural Networks for Joint Depth Estimation and Semantic Segmentation".IEEE TRANSACTIONS ON NEURAL NETWORKS AND LEARNING SYSTEMS 29.11(2018):5655-5666. |
入库方式： OAI收割
来源：自动化研究所
浏览0
下载0
收藏0
其他版本
除非特别说明，本系统中所有内容都受版权保护，并保留所有权利。