C-Cube: Elastic continuous clustering in the cloud

Zhenjie Zhang, Hu Shu, Zhihong Chong, Hua Lu, Yin Yang

Publikation: Bidrag til bog/antologi/rapportKonferencebidrag i proceedingsForskningpeer review

Abstract

Continuous clustering analysis over a data stream reports clustering results incrementally as updates arrive. Such analysis has a wide spectrum of applications, including traffic monitoring and topic discovery on microblogs. A common characteristic of streaming applications is that the amount of workload fluctuates, often in an unpredictable manner. On the other hand, most existing solutions for continuous clustering assume either a central server, or a distributed setting with a fixed number of dedicated servers. In other words, they are not ELASTIC, meaning that they cannot dynamically adapt to the amount of computational resources to the fluctuating workload. Consequently, they incur considerable waste of resources, as the servers are under-utilized when the amount of workload is low. This paper proposes C-Cube, the first elastic approach to continuous streaming clustering. Similar to popular cloud-based paradigms such as MapReduce, C-Cube routes each new record to a processing unit, e.g., a virtual machine, based on its hash value. Each processing unit performs the required computations, and sends its results to a lightweight aggregator. This design enables dynamic adding/removing processing units, as well as replacing faulty ones and re-running their tasks. In addition to elasticity, C-Cube is also effective (in that it provides quality guarantees on the clustering results), efficient (it minimizes the computational workload at all times), and generally applicable to a large class of clustering criteria. We implemented C-Cube in a real system based on Twitter Storm, and evaluated it using real and synthetic datasets. Extensive experimental results confirm our performance claims
OriginalsprogEngelsk
Titel29th IEEE International Conference on Data Engineering, ICDE 2013, Brisbane, Australia, April 8-12, 2013
RedaktørerChristian S. Jensen, Christopher M. Jermaine, Xiaofang Zhou
Antal sider12
ForlagIEEE Computer Society Press
Publikationsdato2013
Sider577-588
ISBN (Trykt)978-1-4673-4908-6
DOI
StatusUdgivet - 2013
Udgivet eksterntJa
Begivenhed29th IEEE International Conference on Data Engineering - Brisbane, Australien
Varighed: 8 apr. 201311 apr. 2013
Konferencens nummer: 29

Konference

Konference29th IEEE International Conference on Data Engineering
Nummer29
Land/OmrådeAustralien
ByBrisbane
Periode08/04/201311/04/2013
NavnProceedings of the International Conference on Data Engineering
ISSN1063-6382

Citer dette