In-Memory Big Data Management and Processing: A Survey

Growing main memory capacity has fueled the development of in-memory big data management and processing. By eliminating disk I/O bottleneck, it is now possible to support interactive data analytics. However, in-memory systems are much more sensitive to other sources of overhead that do not matter in...

Full description

Saved in:
Bibliographic Details
Published inIEEE transactions on knowledge and data engineering Vol. 27; no. 7; pp. 1920 - 1948
Main Authors Zhang, Hao, Chen, Gang, Ooi, Beng Chin, Tan, Kian-Lee, Zhang, Meihui
Format Journal Article
LanguageEnglish
Published IEEE 01.07.2015
Subjects
distributed databases
DRAM
Indexes
Memory management
Optimization
Parallel processing
primary memory
query processing
Random access memory
Registers
relational databases
query processing
DRAM
relational databases
Primary memory
distributed databases
Online AccessGet full text

Cover

More Information
Summary:Growing main memory capacity has fueled the development of in-memory big data management and processing. By eliminating disk I/O bottleneck, it is now possible to support interactive data analytics. However, in-memory systems are much more sensitive to other sources of overhead that do not matter in traditional I/O-bounded disk-based systems. Some issues such as fault-tolerance and consistency are also more challenging to handle in in-memory environment. We are witnessing a revolution in the design of database systems that exploits main memory as its data storage layer. Many of these researches have focused along several dimensions: modern CPU and memory hierarchy utilization, time/space efficiency, parallelism, and concurrency control. In this survey, we aim to provide a thorough review of a wide range of in-memory data management and processing proposals and systems, including both data storage systems and data processing frameworks. We also give a comprehensive presentation of important technology in memory management, and some key factors that need to be considered in order to achieve efficient in-memory data management and processing.
ISSN:1041-4347
1558-2191
DOI:10.1109/TKDE.2015.2427795