Data management for high energy physics experiments — Preliminary proposals

• Published in: Computer physics communications Vol. 45; no. 1; pp. 379 - 383
• Main Authors: Olken, Frank, Loken, Stewart C., Rotem, Doron, Shoshani, Arie, Trippe, Thomas G.
• Format: Journal Article
• Language: English
• Published: Elsevier B.V 01.08.1987
• ISSN: 0010-4655; 1879-2944
• DOI: 10.1016/0010-4655(87)90177-9

Currently HEP experimental data are reduced as they become available. We propose instead a “demand driven” approach to data analysis. Full analysis will be performed only as needed, in response to user queries which specify the subset of events for which reduced data is needed. To support this approach we propose to partition the datasets on the cross product of several trigger inputs, instead of storing the data in chronological order. Queries will be automatically decomposed into a set of requests against several partitions. Indexing, physically clustering the data on the logical partitions, and caching of partitions will be employed for efficiency.