中国科学院机构知识库网格
Chinese Academy of Sciences Institutional Repositories Grid
首页 机构 成果 学者
  
  1. CAS IR Grid
  2. 地理科学与资源研究所
  3. 中国科学院地理科学与资源研究所
GeoSpark SQL: An Effective Framework Enabling Spatial Queries on Spark

文献类型：期刊论文

作者Huang, Zhou1,2,3; Chen, Yiran2,3; Wan, Lin4; Peng, Xia5,6
刊名ISPRS INTERNATIONAL JOURNAL OF GEO-INFORMATION
出版日期2017-09-01
卷号6期号:9页码:20
关键词big data GeoSpark SQL Spark spatial query processing spatial database
ISSN号2220-9964
DOI10.3390/ijgi6090285
通讯作者Peng, Xia(ivy_px@163.com)
英文摘要In the era of big data, Internet-based geospatial information services such as various LBS apps are deployed everywhere, followed by an increasing number of queries against the massive spatial data. As a result, the traditional relational spatial database (e.g., PostgreSQL with PostGIS and Oracle Spatial) cannot adapt well to the needs of large-scale spatial query processing. Spark is an emerging outstanding distributed computing framework in the Hadoop ecosystem. This paper aims to address the increasingly large-scale spatial query-processing requirement in the era of big data, and proposes an effective framework GeoSpark SQL, which enables spatial queries on Spark. On the one hand, GeoSpark SQL provides a convenient SQL interface; on the other hand, GeoSpark SQL achieves both efficient storage management and high-performance parallel computing through integrating Hive and Spark. In this study, the following key issues are discussed and addressed: (1) storage management methods under the GeoSpark SQL framework, (2) the spatial operator implementation approach in the Spark environment, and (3) spatial query optimization methods under Spark. Experimental evaluation is also performed and the results show that GeoSpark SQL is able to achieve real-time query processing. It should be noted that Spark is not a panacea. It is observed that the traditional spatial database PostGIS/PostgreSQL performs better than GeoSpark SQL in some query scenarios, especially for the spatial queries with high selectivity, such as the point query and the window query. In general, GeoSpark SQL performs better when dealing with compute-intensive spatial queries such as the kNN query and the spatial join query.
资助项目National Key Research and Development Program of China[2017YFB0503602] ; National Natural Science Foundation of China[41401449] ; National Natural Science Foundation of China[41501162] ; National Natural Science Foundation of China[41771425] ; Scientific Research Key Program of Beijing Municipal Commission of Education[KM201611417004] ; Beijing Philosophy and Social Science Foundation ; Talent Optimization Program of Beijing Union University ; State Key Laboratory of Resources and Environmental Information System
WOS研究方向Physical Geography ; Remote Sensing
语种英语
WOS记录号WOS:000416386100026
出版者MDPI AG
资助机构National Key Research and Development Program of China ; National Natural Science Foundation of China ; Scientific Research Key Program of Beijing Municipal Commission of Education ; Beijing Philosophy and Social Science Foundation ; Talent Optimization Program of Beijing Union University ; State Key Laboratory of Resources and Environmental Information System
源URL[http://ir.igsnrr.ac.cn/handle/311030/56703]  
专题中国科学院地理科学与资源研究所
通讯作者Peng, Xia
作者单位1.Beijing Adv Innovat Ctr Future Internet Technol, Beijing 100124, Peoples R China
2.Peking Univ, Inst Remote Sensing, Beijing 100871, Peoples R China
3.Peking Univ, GIS, Beijing 100871, Peoples R China
4.China Univ Geosci, Fac Informat Engn, Wuhan 430074, Peoples R China
5.Beijing Union Univ, Inst Tourism, Collaborat Innovat Ctr eTourism, Beijing 100101, Peoples R China
6.Chinese Acad Sci, Inst Geog Sci & Nat Resources Res, State Key Lab Resources & Environm Informat Syst, Beijing 100101, Peoples R China
推荐引用方式
GB/T 7714		 Huang, Zhou,Chen, Yiran,Wan, Lin,et al. GeoSpark SQL: An Effective Framework Enabling Spatial Queries on Spark[J]. ISPRS INTERNATIONAL JOURNAL OF GEO-INFORMATION,2017,6(9):20.
APA Huang, Zhou,Chen, Yiran,Wan, Lin,&Peng, Xia.(2017).GeoSpark SQL: An Effective Framework Enabling Spatial Queries on Spark.ISPRS INTERNATIONAL JOURNAL OF GEO-INFORMATION,6(9),20.
MLA Huang, Zhou,et al."GeoSpark SQL: An Effective Framework Enabling Spatial Queries on Spark".ISPRS INTERNATIONAL JOURNAL OF GEO-INFORMATION 6.9(2017):20.

入库方式： OAI收割

来源：地理科学与资源研究所

浏览0
下载0
收藏0
其他版本

除非特别说明，本系统中所有内容都受版权保护，并保留所有权利。