Model Averaging in the Presence of Structural Uncertainty about Treatment Effects: Influence on Treatment Decision and Expected Value of Information

• Published in: Value in health Vol. 14; no. 2; pp. 205 - 218
• Main Authors: Price, Malcolm J., MSc, PhD, Welton, Nicky J., MSc, PhD, Briggs, Andrew H., MSc, DPhil, Ades, A.E., MSc, PhD
• Format: Journal Article
• Language: English
• Published: United States Elsevier Inc 01.03.2011
• Subjects: Asthma; Asthma - drug therapy; Asthma - economics; Bayes Theorem; Bayesian; Cost-Benefit Analysis - methods; Cost-effectiveness analysis; Decision Making; Evidence synthesis; Humans; Internal Medicine; Markov Chains; Markov model; Model averaging; Models, Theoretical; Monte Carlo Method; Outcome Assessment (Health Care) - methods; Parameterisation; Randomized controlled trial; Randomized Controlled Trials as Topic; Structural uncertainty; Treatment; Uncertainty; Value of information; WinBUGs; Cost-effectiveness analysis; Treatment; WinBUGs; Model averaging; Bayesian; Parameterisation; Markov model; Value of information; Randomized controlled trial; Evidence synthesis; Structural uncertainty; Asthma
• ISSN: 1098-3015; 1524-4733
• DOI: 10.1016/j.jval.2010.08.001

Abstract Background Standard approaches to estimation of Markov models with data from randomized controlled trials tend either to make a judgment about which transition(s) treatments act on, or they assume that treatment has a separate effect on every transition. An alternative is to fit a series of models that assume that treatment acts on specific transitions. Investigators can then choose among alternative models using goodness-of-fit statistics. However, structural uncertainty about any chosen parameterization will remain and this may have implications for the resulting decision and the need for further research. Methods We describe a Bayesian approach to model estimation, and model selection. Structural uncertainty about which parameterization to use is accounted for using model averaging and we developed a formula for calculating the expected value of perfect information (EVPI) in averaged models. Marginal posterior distributions are generated for each of the cost-effectiveness parameters using Markov Chain Monte Carlo simulation in WinBUGS, or Monte-Carlo simulation in Excel (Microsoft Corp., Redmond, WA). We illustrate the approach with an example of treatments for asthma using aggregate-level data from a connected network of four treatments compared in three pair-wise randomized controlled trials. Results The standard errors of incremental net benefit using structured models is reduced by up to eight- or ninefold compared to the unstructured models, and the expected loss attaching to decision uncertainty by factors of several hundreds. Model averaging had considerable influence on the EVPI. Conclusions Alternative structural assumptions can alter the treatment decision and have an overwhelming effect on model uncertainty and expected value of information. Structural uncertainty can be accounted for by model averaging, and the EVPI can be calculated for averaged models.