Evaluating and Combining Physicians' Probabilities of Survival in an Intensive Care Unit

• Published in: Management science Vol. 39; no. 12; pp. 1526 - 1543
• Main Authors: Winkler, Robert L, Poses, Roy M
• Format: Journal Article
• Language: English
• Published: Linthicum, MD INFORMS 01.12.1993; Institute of Management Sciences; Institute for Operations Research and the Management Sciences
• Series: Management Science
• Subjects: Analytical forecasting; Base interest rates; Biological and medical sciences; Calibration; combining probabilities; Comparative analysis; Computerized, statistical medical data processing and models in biomedicine; Critical care; Decision making models; diagnostic probability evaluation; expertise; Health care; Intensive care; Intensive care units; Internships; Judgement; Judgments; Medical sciences; Medical statistics; Physicians; physicians' judgments; Probabilities; Probability; probability assessment; Probability forecasts; Professional responsibilities; Studies; Weather forecasting; Human; Professional experience; Parameter estimation; Statistical analysis; Theory of probabilities; Physician; Medical specialty; Survival; Intensive care unit
• ISSN: 0025-1909; 1526-5501
• DOI: 10.1287/mnsc.39.12.1526

In this paper, probabilities of survival assessed by physicians for patients admitted to an intensive care unit are studied. The probabilities from each of four types of physicians are evaluated on an overall basis and in terms of specific attributes, and the groups are compared. The physicians with the most experience and expertise perform better overall. All four groups appear to be reasonably well calibrated, and the key factor in relative overall performance is the level of discrimination provided by the probabilities. Averages of two, three, and four probabilities for each individual patient are also analyzed. As the number of the probabilities in the average increases, performance improves on average on all dimensions, although the best overall performance is exhibited by a combination of probabilities from the two physician types performing best individually. Some comparisons are made with previous work, and implications for probability assessment and combination in medicine and more generally in other areas of application are discussed. Important characteristics of the study are the fact that it was conducted on-line in a real setting, the involvement of individuals with different levels of expertise, the use of a true predictive situation with a clearly-defined event, the consideration of multiple dimensions of the quality of judgments, and the collection of multiple probabilities for each case to permit the investigation of a variety of possible combinations of probabilities.