Pano-SfMLearner: Self-Supervised Multi-Task Learning of Depth and Semantics in Panoramic Videos
文献类型：期刊论文
| 作者 | Liu, Mengyi3; Wang, Shuhui2; Guo, Yulan1; He, Yuan3; Xue, Hui3 |
| 刊名 | IEEE SIGNAL PROCESSING LETTERS
 |
| 出版日期 | 2021 |
| 卷号 | 28页码:832-836 |
| 关键词 | Depth estimation semantic segmentation pano-ramic video self-supervised learning |
| ISSN号 | 1070-9908 |
| DOI | 10.1109/LSP.2021.3073627 |
| 英文摘要 | With the advent of virtual reality and augment reality applications, omnidirectional imaging and 360 degrees cameras become increasingly popular in many scenarios such as entertainment and autonomous systems. In this paper, we propose a self-supervised framework for multi-task learning on depth, camera motion and semantics frompanoramic videos. Specifically, our method is based on differentiable warping of adjacent views to the target. Two improvements are provided. First, we introduce a view synthesis module based on equirectangular projection to enable direct optimization on panoramic images. Second, we introduce a self-supervised segmentation branch to involve the constraint of semantic consistency for further improvement. Extensive experiments on two 360 degrees video and two 360 degrees image datasets demonstrate that ourmethod outperforms the state-of-the-art and achieves favorable cross-modality performance. |
| 资助项目 | National Natural Science Foundation of China[U20A20185] ; National Natural Science Foundation of China[61972435] ; Natural Science Foundation of Guangdong Province[2019A1515011271] ; Science and Technology Innovation Committee of Shenzhen Municipality[JCYJ20190807152209394] |
| WOS研究方向 | Engineering |
| 语种 | 英语 |
| WOS记录号 | WOS:000648329700002 |
| 出版者 | IEEE-INST ELECTRICAL ELECTRONICS ENGINEERS INC |
| 源URL | [http://119.78.100.204/handle/2XEOYT63/17767]  |
| 专题 | 中国科学院计算技术研究所期刊论文_英文 |
| 通讯作者 | Liu, Mengyi |
| 作者单位 | 1.Sun Yat Sen Univ, Sch Elect & Commun Engn, Guangzhou 510275, Peoples R China 2.Chinese Acad Sci, Inst Comp Technol, Beijing 100190, Peoples R China 3.Alibaba Grp, Beijing 100102, Peoples R China |
| 推荐引用方式 GB/T 7714 | Liu, Mengyi,Wang, Shuhui,Guo, Yulan,et al. Pano-SfMLearner: Self-Supervised Multi-Task Learning of Depth and Semantics in Panoramic Videos[J]. IEEE SIGNAL PROCESSING LETTERS,2021,28:832-836. |
| APA | Liu, Mengyi,Wang, Shuhui,Guo, Yulan,He, Yuan,&Xue, Hui.(2021).Pano-SfMLearner: Self-Supervised Multi-Task Learning of Depth and Semantics in Panoramic Videos.IEEE SIGNAL PROCESSING LETTERS,28,832-836. |
| MLA | Liu, Mengyi,et al."Pano-SfMLearner: Self-Supervised Multi-Task Learning of Depth and Semantics in Panoramic Videos".IEEE SIGNAL PROCESSING LETTERS 28(2021):832-836. |
入库方式： OAI收割
来源：计算技术研究所
浏览0
下载0
收藏0
其他版本
除非特别说明，本系统中所有内容都受版权保护，并保留所有权利。