On the Utility of Graphics Cards to Perform Massively Parallel Simulation of Advanced Monte Carlo Methods

• Published in: Journal of computational and graphical statistics Vol. 19; no. 4; pp. 769 - 789
• Main Authors: Lee, Anthony, Yau, Christopher, Giles, Michael B., Doucet, Arnaud, Holmes, Christopher C.
• Format: Journal Article
• Language: English
• Published: United States Taylor & Francis 01.12.2010; JCGS Management Committee of the American Statistical Association, Institute of Mathematical Statistics, and Interface Foundation of North America; Taylor & Francis Ltd
• Subjects: Algorithms; Architecture; Arithmetic; Computer memory; General purpose computation on graphics processing units; Many-core architecture; Markov analysis; Markov chains; Microprocessors; Monte Carlo methods; Monte Carlo simulation; Outliers and Influential Cases; Parallel computing; Parallel processing; Population-based Markov chain Monte Carlo; Sampling distributions; Sampling methods; Simulation; Stochastic models; Stochastic simulation; Studies
• ISSN: 1061-8600; 1537-2715
• DOI: 10.1198/jcgs.2010.10039

We present a case study on the utility of graphics cards to perform massively parallel simulation of advanced Monte Carlo methods. Graphics cards, containing multiple Graphics Processing Units (GPUs), are self-contained parallel computational devices that can be housed in conventional desktop and laptop computers and can be thought of as prototypes of the next generation of many-core processors. For certain classes of population-based Monte Carlo algorithms they offer massively parallel simulation, with the added advantage over conventional distributed multicore processors that they are cheap, easily accessible, easy to maintain, easy to code, dedicated local devices with low power consumption. On a canonical set of stochastic simulation examples including population-based Markov chain Monte Carlo methods and Sequential Monte Carlo methods, we find speedups from 35- to 500-fold over conventional single-threaded computer code. Our findings suggest that GPUs have the potential to facilitate the growth of statistical modeling into complex data-rich domains through the availability of cheap and accessible many-core computation. We believe the speedup we observe should motivate wider use of parallelizable simulation methods and greater methodological attention to their design. This article has supplementary material online.