Value Uncertainty Analysis in Architecture and Trade Studies

• Published in: IEEE systems journal Vol. 14; no. 4; pp. 5417 - 5428
• Main Authors: Maier, Mark W., Wendoloski, Eric B.
• Format: Journal Article
• Language: English
• Published: New York IEEE 01.12.2020; The Institute of Electrical and Electronics Engineers, Inc. (IEEE)
• Subjects: Analytical models; Baskets; Complexity theory; Computer architecture; Decision analysis; Decision making; Measurement; Satellite constellations; Satellite observation; Stakeholders; systems engineering; Uncertainty; Uncertainty analysis; value-based design
• ISSN: 1932-8184; 1937-9234
• DOI: 10.1109/JSYST.2020.2976717

A key element of architecture studies, analyses-of-alternatives, and related trade studies is a quantitative value model (VM). The VM maps stakeholder preferences for baskets of capabilities to a single number. The mapping ideally has properties that allow for cost-benefit calculations and quantitative trades. An important and difficult element in building a VM is quantifying uncertainty. Uncertainty may stem from uncertainty in stakeholder prioritization, inexact scoring of alternatives against the VM, and the effects of issues outside the VM. This article presents several methods of quantifying uncertainty in the VM resulting from these sources and assessing the sensitivity of preferred alternative architectures to this uncertainty. We discuss methods focused on value uncertainty related to stakeholder consensus, performance scoring, and stakeholder preference uncertainty. Importantly, we include approaches for modeling the strong correlations that exist between alternatives. The methods are applied to a real case study of satellite constellation architecture, the NOAA Satellite Observing System Architecture study, which contains all of the above-mentioned complexities.