Mind the Gap

• Published in: Algorithmica Vol. 81; no. 6; pp. 2123 - 2157
• Main Authors: Amihood Amir, Kopelowitz, Tsvi, Levy, Avivit, Pettie, Seth, Porat, Ely, B. Riva Shalom
• Format: Journal Article
• Language: English
• Published: New York Springer Nature B.V 01.01.2019
• Subjects: Algorithms; Cybersecurity; Dictionaries; Digital signatures; Graph theory; Lower bounds; String matching; Strings; Upper bounds
• ISSN: 0178-4617; 1432-0541
• DOI: 10.1007/s00453-018-0526-2

We examine the complexity of the online Dictionary Matching with One Gap Problem (DMOG) which is the following. Preprocess a dictionary D of d patterns, where each pattern contains a special gap symbol that can match any string, so that given a text that arrives online, a character at a time, we can report all of the patterns from D that are suffixes of the text that has arrived so far, before the next character arrives. In more general versions the gap symbols are associated with bounds determining the possible lengths of matching strings. Online DMOG captures the difficulty in a bottleneck procedure for cyber-security, as many digital signatures of viruses manifest themselves as patterns with a single gap. In this paper, we demonstrate that the difficulty in obtaining efficient solutions for the DMOG problem, even in the offline setting, can be traced back to the infamous 3SUM conjecture. We show a conditional lower bound of Ω(δ(GD)+op) time per text character, where GD is a bipartite graph that captures the structure of D, δ(GD) is the degeneracy of this graph, and op is the output size. Moreover, we show a conditional lower bound in terms of the magnitude of gaps for the bounded case, thereby showing that some known offline upper bounds are essentially optimal. We also provide upper-bounds in terms of the degeneracy for the online DMOG problem. In particular, we introduce algorithms whose time cost depends linearly on δ(GD). Our algorithms make use of graph orientations, together with some additional techniques. These algorithms are of interest for practical cases in which δ(GD) is a small constant. Since δ(GD) can in general be as large as d, and even larger if GD is a multi-graph, we also obtain other solutions adequate for such dense cases.