Predicting the number of fatal soft errors in Los Alamos national laboratory's ASC Q supercomputer

• Published in: IEEE transactions on device and materials reliability Vol. 5; no. 3; pp. 329 - 335
• Main Authors: Michalak, S.E., Harris, K.W., Hengartner, N.W., Takala, B.E., Wender, S.A.
• Format: Magazine Article
• Language: English
• Published: New York IEEE 01.09.2005; The Institute of Electrical and Electronics Engineers, Inc. (IEEE)
• Subjects: Computation; Computational modeling; Computer errors; Cosmic-ray-induced neutron; Crashes; Elevated; Error correction codes; Failure; Fatal; Laboratories; life estimation; Life testing; linear accelerators; memory testing; neutron beam; neutron radiation effects; neutron-induced soft error; Neutrons; Random access memory; Runtime; Semiconductor device testing; semiconductor-device radiation effects; single-event upset; Soft errors; soft-error rate; static random access memory (SRAM) chips; Strategy; Supercomputers
• DOI: 10.1109/TDMR.2005.855685
• ISSN: 1530-4388

Early in the deployment of the Advanced Simulation and Computing (ASC) Q supercomputer, a higher-than-expected number of single-node failures was observed. The elevated rate of single-node failures was hypothesized to be caused primarily by fatal soft errors, i.e., board-level cache (B-cache) tag (BTAG) parity errors caused by cosmic-ray-induced neutrons that led to node crashes. A series of experiments was undertaken at the Los Alamos Neutron Science Center (LANSCE) to ascertain whether fatal soft errors were indeed the primary cause of the elevated rate of single-node failures. Observed failure data from Q are consistent with the results from some of these experiments. Mitigation strategies have been developed, and scientists successfully use Q for large computations in the presence of fatal soft errors and other single-node failures.