Dynamic Space Efficient Hashing

• Published in: Algorithmica Vol. 81; no. 8; pp. 3162 - 3185
• Main Authors: Maier, Tobias, Sanders, Peter, Walzer, Stefan
• Format: Journal Article
• Language: English
• Published: New York Springer US 01.08.2019; Springer Nature B.V
• Subjects: Algorithm Analysis and Problem Complexity; Algorithms; Buckets; Computer Science; Computer Systems Organization and Communication Networks; Data Structures and Information Theory; Inhomogeneity; Load; Lower bounds; Mathematics of Computing; Theory of Computation; Virtual memory systems; Dynamic data structures; Cuckoo hashing; Closed hashing; Open addressing; Space efficiency
• ISSN: 0178-4617; 1432-0541
• DOI: 10.1007/s00453-019-00572-x

We consider space efficient hash tables that can grow and shrink dynamically and are always highly space efficient, i.e., their space consumption is always close to the lower bound even while growing and when taking into account storage that is only needed temporarily. None of the traditionally used hash tables have this property. We show how known approaches like linear probing and bucket cuckoo hashing can be adapted to this scenario by subdividing them into many subtables or using virtual memory overcommitting. However, these rather straightforward solutions suffer from slow amortized insertion times due to frequent reallocation in small increments. Our main result is Dynamic Space Efficient Cuckoo Table (DySECT ) which avoids these problems. DySECT consists of many subtables which grow by doubling their size. The resulting inhomogeneity in subtable sizes is counterbalanced by the flexibility available in bucket cuckoo hashing where each element can go to several buckets each of which containing several cells. Experiments indicate that DySECT works well with loads up to 98%. With up to 1.9 times better performance than the next best solution. Additionally, we give a tight theoretical analysis for the possible load threshold of DySECT, i.e., a bound where with high probability the table can be filled up to that load but not above said load. This load also matches our experimental findings.