BV-CEI: A Compact Implementation of Interval Query Indexing Algorithm

• Published in: 2021 IEEE 15th International Conference on Big Data Science and Engineering (BigDataSE) pp. 80 - 86
• Main Authors: Lu, Yuhai, Zhang, Chunyan, Cao, Cong, Liu, Yanbing, Tan, Jianlong
• Format: Conference Proceeding
• Language: English
• Published: IEEE 01.10.2021
• Subjects: Big Data; Conferences; continual query; Costs; Data models; data stream model; Filtering; interval index; interval query; Intrusion detection; query index; Real-time systems
• DOI: 10.1109/BigDataSE53435.2021.00021

The data stream model has been widely used to process streaming data, including intrusion detection system and web content filtering system in the network and security field and so on. Interval query index is a key technology used in the data stream system, which can be divided into direct interval query index and indirect interval query index. The search speed of indirect interval query index algorithm can better meet the real-time demand of data stream system. Among which, CEI has the best search performance, but its storage cost is too high. In this article, we proposed a compact implementation of interval query indexing algorithm CEI based on bit vector (named BV-CEI) to reduce the storage cost of interval index structure. We employed the index structure based on bit vector instead of the index structure based on pointer, and designed a quick rank operation to accelerate the query speed of bit vector. We evaluate the experiment on random data. Compared with CEI, BV-CEI can significantly reduce the storage cost by about 26 times, meanwhile the searching speed can be kept at the same level as the CEI.