Massive point cloud data management: Design, implementation and execution of a point cloud benchmark

• Published in: Computers & graphics Vol. 49; pp. 92 - 125
• Main Authors: van Oosterom, Peter, Martinez-Rubi, Oscar, Ivanova, Milena, Horhammer, Mike, Geringer, Daniel, Ravada, Siva, Tijssen, Theo, Kodde, Martin, Gonçalves, Romulo
• Format: Journal Article
• Language: English
• Published: Elsevier Ltd 01.06.2015
• Subjects: Benchmark; Benchmarking; Data management; DBMS; Density; Mathematical models; Organizations; Parallel processing; Point cloud data; Space filling curve; Tables (data); Three dimensional models; Vario-scale; Benchmark; Parallel processing; DBMS; Point cloud data; Space filling curve; Vario-scale
• ISSN: 0097-8493; 1873-7684
• DOI: 10.1016/j.cag.2015.01.007

Point cloud data are important sources for 3D geo-information. An inventory of the point cloud data management user requirements has been compiled using structured interviews with users from different background: government, industry and academia. Based on these requirements a benchmark has been developed to compare various point cloud data management solutions with regard to functionality and performance. The main test dataset is the second national height map of the Netherlands, AHN2, with 6–10 samples for every square meter of the country, resulting in 640 billion points. At the database level, a data storage model based on grouping the points in blocks is available in Oracle and PostgreSQL. This model is compared with the ‘flat table’ model, where each point is stored in a table row, in Oracle, PostgreSQL and the column-store MonetDB. In addition, the commonly used file-based solution Rapidlasso LAStools is used for comparison with the database solutions. The results of executing the benchmark on different platforms are presented as obtained during the increasingly challenging stages with more functionality and more data: mini (20 million points), medium (20 billion points), and full benchmark (the complete AHN2). During the design, the implementation and the execution of the benchmarks, a number of point cloud data management improvements were proposed and partly tested: Morton/Hilbert code for ordering data (especially in flat model), two algorithms for parallel query execution, and a unique vario-scale LoD data organization avoiding the density jumps of the well-known discrete LoD data organizations. [Display omitted] •Design of point cloud benchmark based on requirements from different groups of users within government, industry and academia.•Analysing various data management systems: PostgreSQL, MonetDB, Oracle, and LAStools.•New techniques for point cloud management: Morton code and Morton-ranges, algorithms for parallel query, and vario-LoD organization.