On the performance and convergence of distributed stream processing via approximate fault tolerance
文献类型：期刊论文
| 作者 | Cheng, Zhinan1; Huang, Qun2; Lee, Patrick P. C.1 |
| 刊名 | VLDB JOURNAL
 |
| 出版日期 | 2019-10-01 |
| 卷号 | 28期号:5页码:821-846 |
| 关键词 | Distributed stream processing Approximate fault tolerance Online learning |
| ISSN号 | 1066-8888 |
| DOI | 10.1007/s00778-019-00565-w |
| 英文摘要 | Fault tolerance is critical for distributed stream processing systems, yet achieving error-free fault tolerance often incurs substantial performance overhead. We present AF-Stream, a distributed stream processing system that addresses the trade-off between performance and accuracy in fault tolerance. AF-Stream builds on a notion called approximate fault tolerance, whose idea is to mitigate backup overhead by adaptively issuing backups, while ensuring that the errors upon failures are bounded with theoretical guarantees. Specifically, AF-Stream allows users to specify bounds on both the state divergence and the loss of non-backup streaming items. It issues state and item backups only when the bounds are reached. Our AF-Stream design provides an extensible programming model for incorporating general streaming algorithms as well as exports only few threshold parameters for configuring approximation fault tolerance. Furthermore, we formally prove that AF-Stream preserves high algorithm-specific accuracy of streaming algorithms, and in particular the convergence guarantees of online learning. Experiments show that AF-Stream maintains high performance (compared to no fault tolerance) and high accuracy after multiple failures (compared to no failures) under various streaming algorithms. |
| 资助项目 | Research Grants Council of Hong Kong[GRF 14204017] ; Innovation and Technology Commission of Hong Kong[ITS/113/14] ; Huawei Technologies[HF2017060008] ; National Natural Science Foundation of China[61802365] ; CAS Pioneer Hundred Talents Program |
| WOS研究方向 | Computer Science |
| 语种 | 英语 |
| WOS记录号 | WOS:000490007100008 |
| 出版者 | SPRINGER |
| 源URL | [http://119.78.100.204/handle/2XEOYT63/4635]  |
| 专题 | 中国科学院计算技术研究所期刊论文_英文 |
| 通讯作者 | Huang, Qun |
| 作者单位 | 1.Chinese Univ Hong Kong, Dept Comp Sci & Engn, Sha Tin, Hong Kong, Peoples R China 2.Univ Chinese Acad Sci, Chinese Acad Sci, Inst Comp Technol, State Key Lab Comp Architecture, Beijing, Peoples R China |
| 推荐引用方式 GB/T 7714 | Cheng, Zhinan,Huang, Qun,Lee, Patrick P. C.. On the performance and convergence of distributed stream processing via approximate fault tolerance[J]. VLDB JOURNAL,2019,28(5):821-846. |
| APA | Cheng, Zhinan,Huang, Qun,&Lee, Patrick P. C..(2019).On the performance and convergence of distributed stream processing via approximate fault tolerance.VLDB JOURNAL,28(5),821-846. |
| MLA | Cheng, Zhinan,et al."On the performance and convergence of distributed stream processing via approximate fault tolerance".VLDB JOURNAL 28.5(2019):821-846. |
入库方式： OAI收割
来源：计算技术研究所
浏览0
下载0
收藏0
其他版本
除非特别说明，本系统中所有内容都受版权保护，并保留所有权利。