中国科学院机构知识库网格
Chinese Academy of Sciences Institutional Repositories Grid
首页 机构 成果 学者
  
  1. CAS IR Grid
  2. 计算技术研究所
  3. 中国科学院计算技术研究所
  4. 中国科学院计算技术研究所期刊论文
  5. 英文
Load-balancing distributed outer joins through operator decomposition

文献类型：期刊论文

作者Cheng, Long2; Kotoulas, Spyros3; Liu, Qingzhi4; Wang, Ying1
刊名JOURNAL OF PARALLEL AND DISTRIBUTED COMPUTING
出版日期2019-10-01
卷号132页码:21-35
ISSN号0743-7315
关键词Distributed join Outer join Data skew Load balancing Spark
DOI10.1016/j.jpdc.2019.05.008
英文摘要High-performance data analytics largely relies on being able to efficiently execute various distributed data operators such as distributed joins. So far, large amounts of join methods have been proposed and evaluated in parallel and distributed environments. However, most of them focus on inner joins, and there is little published work providing the detailed implementations and analysis of outer joins. In this work, we present POPI (Partial Outer join & Partial Inner join), a novel method to load-balance large parallel outer joins by decomposing them into two operations: a large outer join over data that does not present significant skew in the input and an inner join over data presenting significant skew. We present the detailed implementation of our approach and show that POPI is implementable over a variety of architectures and underlying join implementations. Moreover, our experimental evaluation over a distributed memory platform also demonstrates that the proposed method is able to improve outer join performance under varying data skew and present excellent load-balancing properties, compared to current approaches. (C) 2019 Elsevier Inc. All rights reserved.
资助项目European Union's Horizon 2020 research and innovation programme under the Marie Sklodowska-Curie grant[799066]
WOS研究方向Computer Science
语种英语
出版者ACADEMIC PRESS INC ELSEVIER SCIENCE
WOS记录号WOS:000476580400003
源URL[http://119.78.100.204/handle/2XEOYT63/4459]  
专题中国科学院计算技术研究所期刊论文_英文
通讯作者Cheng, Long
作者单位1.Chinese Acad Sci, Inst Comp Technol, Beijing, Peoples R China
2.Univ Coll Dublin, Sch Comp Sci, Dublin, Ireland
3.IBM Res, Dublin, Ireland
4.Eindhoven Univ Technol, Dept Math & Comp Sci, Eindhoven, Netherlands
推荐引用方式
GB/T 7714		 Cheng, Long,Kotoulas, Spyros,Liu, Qingzhi,et al. Load-balancing distributed outer joins through operator decomposition[J]. JOURNAL OF PARALLEL AND DISTRIBUTED COMPUTING,2019,132:21-35.
APA Cheng, Long,Kotoulas, Spyros,Liu, Qingzhi,&Wang, Ying.(2019).Load-balancing distributed outer joins through operator decomposition.JOURNAL OF PARALLEL AND DISTRIBUTED COMPUTING,132,21-35.
MLA Cheng, Long,et al."Load-balancing distributed outer joins through operator decomposition".JOURNAL OF PARALLEL AND DISTRIBUTED COMPUTING 132(2019):21-35.

入库方式： OAI收割

来源：计算技术研究所

浏览0
下载0
收藏0
其他版本

除非特别说明，本系统中所有内容都受版权保护，并保留所有权利。