An Overview of the Sirocco Parallel Storage System

• Published in: High Performance Computing pp. 121 - 129
• Main Authors: Curry, Matthew L., Ward, H. Lee, Danielson, Geoff, Lofstead, Jay
• Format: Book Chapter
• Language: English
• Published: Cham Springer International Publishing
• Series: Lecture Notes in Computer Science
• Subjects: High performance computing; I/O; Parallel file systems
• ISBN: 9783319460789; 3319460781
• ISSN: 0302-9743; 1611-3349
• DOI: 10.1007/978-3-319-46079-6_9

Sirocco is a massively parallel, high performance storage system that breaks from the classical Zebra-style file system design paradigm. Its architecture is inspired by peer-to-peer and victim-cache architectures, and emphasizes client-to-client coordination, low server-side coupling, and free data movement and placement. By leveraging these ideas, Sirocco natively supports automatic migration between several media types, including RAM, flash, disk, and archival storage. Sirocco provides advanced storage interfaces, enabling clients to efficiently use key-value storage or block-based storage through a single interface. It also provides several levels of transactional data updates, up to and including ACID-compliant updates across several objects. Further support is provided for concurrency control, enabling greater performance during safe concurrent modification. By pioneering these and other techniques, Sirocco is well-poised to fulfill a need for a massively scalable, write-optimized storage system. This paper provides an overview of Sirocco’s current system design.