CERN data services for LHC computing

• Published in: Journal of physics. Conference series Vol. 898; no. 6; pp. 62028 - 62035
• Main Authors: Espinal, X, Bocchi, E, Chan, B, Fiorot, A, Iven, J, Lo Presti, G, Lopez, J, Gonzalez, H, Lamanna, M, Mascetti, L, Moscicki, J, Pace, A, Peters, A, Ponce, S, Rousseau, H, van der Ster, D
• Format: Journal Article
• Language: English
• Published: Bristol IOP Publishing 01.10.2017
• Subjects: CERN; Data analysis; Experiments; Flexible components; Infrastructure; Physics; Workload
• ISSN: 1742-6588; 1742-6596
• DOI: 10.1088/1742-6596/898/6/062028

Dependability, resilience, adaptability and efficiency. Growing requirements require tailoring storage services and novel solutions. Unprecedented volumes of data coming from the broad number of experiments at CERN need to be quickly available in a highly scalable way for large-scale processing and data distribution while in parallel they are routed to tape for long-term archival. These activities are critical for the success of HEP experiments. Nowadays we operate at high incoming throughput (14GB/s during 2015 LHC Pb-Pb run and 11PB in July 2016) and with concurrent complex production work-loads. In parallel our systems provide the platform for the continuous user and experiment driven work-loads for large-scale data analysis, including end-user access and sharing. The storage services at CERN cover the needs of our community: EOS and CASTOR as a large-scale storage; CERNBox for end-user access and sharing; Ceph as data back-end for the CERN OpenStack infrastructure, NFS services and S3 functionality; AFS for legacy distributed-file-system services. In this paper we will summarise the experience in supporting LHC experiments and the transition of our infrastructure from static monolithic systems to flexible components providing a more coherent environment with pluggable protocols, tuneable QoS, sharing capabilities and fine grained ACLs management while continuing to guarantee dependable and robust services.