Effectiveness and predictability of in-network storage cache for scientific workflows

• Published in: arXiv.org
• Main Authors: Sim, Caitlin, Wu, Kesheng, Sim, Alex, Monga, Inder, Chin Guok, Wurthwein, Frank, Davila, Diego, Newman, Harvey, Balcas, Justas
• Format: Paper Journal Article
• Language: English
• Published: Ithaca Cornell University Library, arXiv.org 20.07.2023
• Subjects: Communications traffic; Computer Science - Distributed, Parallel, and Cluster Computing; Computer Science - Learning; Computer Science - Networking and Internet Architecture; Data storage; Effectiveness; Machine learning; Network latency; Provisioning; Reduction; Resource allocation; Traffic volume; Wide area networks

Large scientific collaborations often have multiple scientists accessing the same set of files while doing different analyses, which create repeated accesses to the large amounts of shared data located far away. These data accesses have long latency due to distance and occupy the limited bandwidth a...