中国科学院机构知识库网格
Chinese Academy of Sciences Institutional Repositories Grid
首页 机构 成果 学者
  
  1. CAS IR Grid
  2. 计算技术研究所
  3. 中国科学院计算技术研究所
  4. 中国科学院计算技术研究所期刊论文
  5. 英文
i(2)MapReduce: Incremental MapReduce for Mining Evolving Big Data

文献类型：期刊论文

作者Zhang, Yanfeng2; Chen, Shimin1; Wang, Qiang3; Yu, Ge3
刊名IEEE TRANSACTIONS ON KNOWLEDGE AND DATA ENGINEERING
出版日期2015-07-01
卷号27期号:7页码:1906-1919
关键词Incremental processing MapReduce iterative computation big data
ISSN号1041-4347
DOI10.1109/TKDE.2015.2397438
英文摘要As new data and updates are constantly arriving, the results of data mining applications become stale and obsolete over time. Incremental processing is a promising approach to refreshing mining results. It utilizes previously saved states to avoid the expense of re-computation from scratch. In this paper, we propose i(2)MapReduce, a novel incremental processing extension to MapReduce, the most widely used framework for mining big data. Compared with the state-of-the-art work on Incoop, i(2)MapReduce (i) performs key-value pair level incremental processing rather than task level re-computation, (ii) supports not only one-step computation but also more sophisticated iterative computation, which is widely used in data mining applications, and (iii) incorporates a set of novel techniques to reduce I/O overhead for accessing preserved fine-grain computation states. We evaluate i(2)MapReduce using a one-step algorithm and four iterative algorithms with diverse computation characteristics. Experimental results on Amazon EC2 show significant performance improvements of i(2)MapReduce compared to both plain and iterative MapReduce performing re-computation.
资助项目National Natural Science Foundation of China[61300023] ; National Natural Science Foundation of China[61433008] ; National Natural Science Foundation of China[61272179] ; Fundamental Research Funds for the Central Universities[N141605001] ; Fundamental Research Funds for the Central Universities[N120816001] ; China Mobile Fund[MCM20125021] ; MOE-Intel Special Fund of Information Technology[MOE-INTEL-2012-06] ; CAS Hundred Talents program ; NSFC Innovation Research Group[61221062]
WOS研究方向Computer Science ; Engineering
语种英语
WOS记录号WOS:000355937800013
出版者IEEE COMPUTER SOC
源URL[http://119.78.100.204/handle/2XEOYT63/9733]  
专题中国科学院计算技术研究所期刊论文_英文
通讯作者Zhang, Yanfeng
作者单位1.Chinese Acad Sci, Inst Comp Technol, State Key Lab Comp Architecture, Beijing 100864, Peoples R China
2.Northeastern Univ, Ctr Comp, Shenyang 110819, Peoples R China
3.Northeastern Univ, Shenyang 110819, Peoples R China
推荐引用方式
GB/T 7714		 Zhang, Yanfeng,Chen, Shimin,Wang, Qiang,et al. i(2)MapReduce: Incremental MapReduce for Mining Evolving Big Data[J]. IEEE TRANSACTIONS ON KNOWLEDGE AND DATA ENGINEERING,2015,27(7):1906-1919.
APA Zhang, Yanfeng,Chen, Shimin,Wang, Qiang,&Yu, Ge.(2015).i(2)MapReduce: Incremental MapReduce for Mining Evolving Big Data.IEEE TRANSACTIONS ON KNOWLEDGE AND DATA ENGINEERING,27(7),1906-1919.
MLA Zhang, Yanfeng,et al."i(2)MapReduce: Incremental MapReduce for Mining Evolving Big Data".IEEE TRANSACTIONS ON KNOWLEDGE AND DATA ENGINEERING 27.7(2015):1906-1919.

入库方式： OAI收割

来源：计算技术研究所

浏览0
下载0
收藏0
其他版本

除非特别说明，本系统中所有内容都受版权保护，并保留所有权利。