Theory-driven or process-driven prediction? Epistemological challenges of big data analytics

• Published in: Journal of big data Vol. 4; no. 1; pp. 1 - 20
• Main Authors: Elragal, Ahmed, Klischewski, Ralf
• Format: Journal Article
• Language: English
• Published: Cham Springer International Publishing 23.06.2017; Springer Nature B.V; SpringerOpen
• Subjects: Analytics; Big Data; Big Data Analytics; Communications Engineering; Computational Science and Engineering; Computer Science; Consolidation; Data analysis; Data management; Data Mining and Knowledge Discovery; Data processing; Database Management; Epistemological challenges; Epistemology; Information Storage and Retrieval; Information Systems; Information systems theories; Informationssystem; Mathematical analysis; Mathematical Applications in Computer Science; Networks; Predictive research; Preprocessing; Studies; Big data analytics; Information systems theories; Epistemological challenges; Predictive research
• ISSN: 2196-1115; 2196-1115
• DOI: 10.1186/s40537-017-0079-2

Most scientists are accustomed to make predictions based on consolidated and accepted theories pertaining to the domain of prediction. However, nowadays big data analytics (BDA) is able to deliver predictions based on executing a sequence of data processing while seemingly abstaining from being theoretically informed about the subject matter. This paper discusses how to deal with the shift from theory-driven to process-driven prediction through analyzing the BDA steps and identifying the epistemological challenges and various needs of theoretically informing BDA throughout data acquisition, preprocessing, analysis, and interpretation. We suggest a theory-driven guidance for the BDA process including acquisition, pre-processing, analytics and interpretation. That is, we propose—in association with these BDA process steps—a lightweight theory-driven approach in order to safeguard the analytics process from epistemological pitfalls. This study may serve as a guideline for researchers and practitioners to consider while conducting future big data analytics. Graphical abstract Epistemological challenges of big data analytics.