A Theoretical Model for Assessing Information Validity from Multiple Observers

Assessing the validity of observational data from multiple sources (i.e. the likelihood that an observation is true) is an especially challenging task. There are several principles or constructs in the social sciences that provide a framework for assessing validity in qualitative research. Similarly...

Full description

Saved in:
Bibliographic Details
Published in2019 IEEE Conference on Cognitive and Computational Aspects of Situation Management (CogSIMA) pp. 52 - 58
Main Authors Dorton, Stephen L., Frommer, Ian D., Garrison, Teena M.
Format Conference Proceeding
LanguageEnglish
Published IEEE 01.04.2019
Subjects
Argumentation
Auditory system
Collective intelligence
Concurrence
Crowdsourcing
HUMINT
Information Validity
Observational Data
Observers
OSINT
Reliability
Social sciences
Task analysis
Online AccessGet full text

Cover

More Information
Summary:Assessing the validity of observational data from multiple sources (i.e. the likelihood that an observation is true) is an especially challenging task. There are several principles or constructs in the social sciences that provide a framework for assessing validity in qualitative research. Similarly, methods and measures exist to assess the agreeability and reliability of multiple observers; however, they are limited to assessing how much two or more people agree on a set of multiple observations-not the validity of an individual observation. We generate a model and component axioms based on principles in the social sciences to enable a crowdsourcing approach to rapidly quantify information validity from multiple observers. This research has implications in conducting qualitative research, as well as analyzing information gleaned from Human Intelligence (HUMINT) and Open Source Intelligence (OSINT) applications. Limitations to this model and areas of future research are also discussed.
ISSN:2379-1675
DOI:10.1109/COGSIMA.2019.8724242