中国科学院机构知识库网格
Chinese Academy of Sciences Institutional Repositories Grid
首页 机构 成果 学者
  
  1. CAS IR Grid
  2. 地理科学与资源研究所
  3. 中国科学院地理科学与资源研究所

Multi-modal fusion of satellite and street-view images for urban village classification based on a dual-branch deep neural network

文献类型：期刊论文

作者Chen, Boan2; Feng, Quanlong2,4; Niu, Bowen2; Yan, Fengqin4; Gao, Bingbo2; Yang, Jianyu2; Gong, Jianhua1; Liu, Jiantao3
刊名INTERNATIONAL JOURNAL OF APPLIED EARTH OBSERVATION AND GEOINFORMATION
出版日期2022-05-01
卷号109页码:15
关键词Remote sensing Street-view Deep learning Urban village
ISSN号1569-8432
DOI10.1016/j.jag.2022.102794
通讯作者Feng, Quanlong(fengql@cau.edu.cn)
英文摘要With the rapid urbanization process in China, numerous urban villages have been appeared, which are surrounded by the newly-built urban blocks. Due to the high population density, poor hygiene, chaotic waste discharge, and inadequate public facilities, urban villages have many negative impacts on both urban environment and management. The objective of this study is to propose a dual-branch deep learning model for multi modal satellite and street-view data fusion to detect urban villages in Beijing, Tianjin and Shijiazhuang, which are the core cities of Jing-Jin-Ji region of China. Specifically, the proposed model consists of a satellite branch, a street-view branch and a gated-fusion module. As for the satellite branch, a Trans-MDCNN (multi-scale dilated convolutional neural network) is proposed to learn multi-level local features and global contextual features from high resolution satellite imagery, while for the street-view branch, an MVRAN (multi-view recurrent attention network) is constructed to learn and fuse multi-angle features from street-view images. A gated-fusion module is designed to aggregate the important features of the dual-branches. Experimental results indicate that the proposed model has achieved good performance with an overall accuracy (OA) of 92.61%. Ablation study shows that compared with satellite data alone, the integration of street-view images could increase the OA by about 2%. Besides, 1-D feature fusion outperforms its 2-D counterpart and the classic feature concatenation method. The proposed model also yields a better performance than other deep learning models. Finally, the dataset of this study, (SUV)-U-2 (Satellite & Street-view images for Urban Village classification), is publicly available: https://doi.org/10.11922/sciencedb.01410.
WOS关键词SEMANTIC SEGMENTATION ; INFORMAL SETTLEMENTS ; SLUMS
资助项目State Key Laboratory of Re-sources and Environmental Information System, National Natural Sci-ence Foundation of China[42001367] ; State Key Laboratory of Re-sources and Environmental Information System, National Natural Sci-ence Foundation of China[42171113]
WOS研究方向Remote Sensing
语种英语
WOS记录号WOS:000802151000002
出版者ELSEVIER
资助机构State Key Laboratory of Re-sources and Environmental Information System, National Natural Sci-ence Foundation of China
源URL[http://ir.igsnrr.ac.cn/handle/311030/178634]  
专题中国科学院地理科学与资源研究所
通讯作者Feng, Quanlong
作者单位1.Chinese Acad Sci, Aerosp Informat Res Inst, Natl Engn Res Ctr Geoinformat, Beijing 100101, Peoples R China
2.China Agr Univ, Coll Land Sci & Technol, Beijing 100193, Peoples R China
3.Shandong Jianzhu Univ, Sch Surveying & Geoinformat, Jinan 250101, Shandong, Peoples R China
4.Chinese Acad Sci, Inst Geog Sci & Nat Resources Res, State Key Lab Resources & Environm Informat Syst, Beijing 100101, Peoples R China
推荐引用方式
GB/T 7714		 Chen, Boan,Feng, Quanlong,Niu, Bowen,et al.

Multi-modal fusion of satellite and street-view images for urban village classification based on a dual-branch deep neural network

[J]. INTERNATIONAL JOURNAL OF APPLIED EARTH OBSERVATION AND GEOINFORMATION,2022,109:15.
APA Chen, Boan.,Feng, Quanlong.,Niu, Bowen.,Yan, Fengqin.,Gao, Bingbo.,...&Liu, Jiantao.(2022).

Multi-modal fusion of satellite and street-view images for urban village classification based on a dual-branch deep neural network

.INTERNATIONAL JOURNAL OF APPLIED EARTH OBSERVATION AND GEOINFORMATION,109,15.
MLA Chen, Boan,et al."

Multi-modal fusion of satellite and street-view images for urban village classification based on a dual-branch deep neural network

".INTERNATIONAL JOURNAL OF APPLIED EARTH OBSERVATION AND GEOINFORMATION 109(2022):15.

入库方式： OAI收割

来源：地理科学与资源研究所

浏览0
下载0
收藏0
其他版本

除非特别说明，本系统中所有内容都受版权保护，并保留所有权利。