Lightweight monitoring of distributed streams

Arnon Lazerson, Daniel Keren, Assaf Schuster

Research output: Contribution to journalArticlepeer-review

Abstract

As data becomes dynamic, large, and distributed, there is increasing demand for what have become known as distributed stream algorithms. Since continuously collecting the data to a central server and processing it there is infeasible, a common approach is to define local conditions at the distributed nodes, such that-as long as they are maintained-some desirable global condition holds. Previous methods derived local conditions focusing on communication efficiency. While proving very useful for reducing the communication volume, these local conditions often suffer from heavy computational burden at the nodes. The computational complexity of the local conditions affects both the runtime and the energy consumption. These are especially critical for resource-limited devices like smartphones and sensor nodes. Such devices are becoming more ubiquitous due to the recent trend toward smart cities and the Internet of Things. To accommodate for high data rates and limited resources of these devices, it is crucial that the local conditions be quickly and efficiently evaluated. Here we propose a novel approach, designated CB (for Convex/Concave Bounds). CB defines local conditions using suitably chosen convex and concave functions. Lightweight and simple, these local conditions can be rapidly checked on the fly. CB's superiority over the state-of-the-art is demonstrated in its reduced runtime and power consumption, by up to six orders of magnitude in some cases. As an added bonus, CB also reduced communication overhead in all the tested application scenarios.

Original languageEnglish
Article number3226113
JournalACM Transactions on Database Systems
Volume43
Issue number2
DOIs
StatePublished - Jul 2018

Bibliographical note

Funding Information:
The research leading to these results has received funding from the [European Union’s] Seventh Framework Programme [FP7-ICT-2013-11] under Grant Agreement No. 619491 and No. 619435 and from the European Commission Horizon 2020-the Framework Programme for Research and Innovation (2014-2020) under Grant Agreement No. 688380. Authors’ addresses: A. Lazerson, Technion – Israel Institute of Technology, Haifa 32000, Israel; email: lazerson@ cs.technion.ac.il; D. Keren, University of Haifa, Haifa 31905, Israel; email: dkeren@cs.haifa.ac.il; A. Schuster, Technion – Israel Institute of Technology, Haifa 32000, Israel; email: assaf@cs.technion.ac.il. Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than ACM must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from permissions@acm.org. © 2018 ACM 0362-5915/2018/07-ART9 $15.00 https://doi.org/10.1145/3226113

Publisher Copyright:
©2018 ACM.

Keywords

  • Continuous Distributed Monitoring
  • Disributed Stream Mining

ASJC Scopus subject areas

  • Information Systems

Fingerprint

Dive into the research topics of 'Lightweight monitoring of distributed streams'. Together they form a unique fingerprint.

Cite this