GVoS: A General System for Near-Duplicate Video-Related Applications on Storm

Jiawei Jiang, Yunhai Tong, Hua Lu, Bin Cui, Kai Lei, Lele Yu

Publikation: Bidrag til tidsskriftTidsskriftartikelForskningpeer review

Abstract

The exponential increase of online videos greatly enriches the life of users but also brings huge numbers of near-duplicate videos (NDVs) that seriously challenge the video websites. The video websites entail NDV-related applications such as detection of copyright violation, video monitoring, video re-ranking, and video recommendation. Since these applications adopt different features and different processing procedures due to diverse scenarios, constructing separate and special-purpose systems for them incurs considerable costs on design, implementation, and maintenance. In this article, we propose a general NDV system on Storm (GVoS)—a popular distributed real-time stream processing platform—to simultaneously support a wide variety of video applications. The generality of GVoS is achieved in two aspects. First, we extract the reusable components from various applications. Second, we conduct the communication between components via a mechanism called Stream Shared Message (SSM) that contains the video-related data. Furthermore, we present an algorithm to reduce the size of SSM in order to avoid the data explosion and decrease the network latency. The experimental results demonstrate that GVoS can achieve performance almost the same as the customized systems. Meanwhile, GVoS accomplishes remarkably higher systematic versatility and efficiently facilitates the development of various NDV-related applications.
OriginalsprogEngelsk
Artikelnummer3
TidsskriftA C M Transactions on Information Systems
Vol/bind36
Udgave nummer1
Sider (fra-til)1-36
ISSN1046-8188
DOI
StatusUdgivet - jun. 2017
Udgivet eksterntJa

Citer dette