Legion: Lessons Learned Building a Grid Operating System

• Published in: Proceedings of the IEEE Vol. 93; no. 3; pp. 589 - 603
• Main Authors: Grimshaw, A.S., Natrajan, A.
• Format: Journal Article
• Language: English
• Published: New York IEEE 01.03.2005; The Institute of Electrical and Electronics Engineers, Inc. (IEEE)
• Subjects: Application software; Architecture; Autonomous; Bleeding; Buildings; Commercialization; Complexity; Computation; Contracts; Distributed object system; grid; grid architecture; Grid computing; grid design philosophy; large-scale distributed system; metaoperating systems; metasystems; Middleware; Operating systems; Resource management; Solids; US Department of Energy; Virtual machining
• ISSN: 0018-9219; 1558-2256
• DOI: 10.1109/JPROC.2004.842764

Legion was the first integrated grid middleware architected from first principles to address the complexity of grid environments. Just as a traditional operating system provides an abstract interface to the underlying physical resources of a machine, Legion was designed to provide a powerful virtual machine interface layered over the distributed, heterogeneous, autonomous, and fault-prone physical and logical resources that constitute a grid. We believe that without a solid, integrated, operating system-like grid middleware, grids will fail to cross the chasm from bleeding-edge supercomputing users to more mainstream computing. This work provides an overview of the architectural principles that drove Legion, a high-level description of the system with complete references to more detailed explanations, and the history of Legion from first inception in August 1993 through commercialization. We present a number of important lessons, both technical and sociological, learned during the course of developing and deploying Legion.