Data science, big data and statistics

This article analyzes how Big Data is changing the way we learn from observations. We describe the changes in statistical methods in seven areas that have been shaped by the Big Data-rich environment: the emergence of new sources of information; visualization in high dimensions; multiple testing pro...

Full description

Saved in:
Bibliographic Details
Published inTest (Madrid, Spain) Vol. 28; no. 2; pp. 289 - 329
Main Authors Galeano, Pedro, Peña, Daniel
Format Journal Article
LanguageEnglish
Published Berlin/Heidelberg Springer Berlin Heidelberg 01.06.2019
Springer Nature B.V
Subjects
Big Data
Data analysis
Data management
Data science
Economics
Finance
Insurance
Invited Paper
Machine learning
Management
Mathematics and Statistics
Statistical methods
Statistical models
Statistical Theory and Methods
Statistics
Statistics for Business
Multivariate data
Statistical learning
Network analysis
62A01
Machine learning
Time series
Sparse model selection
62H99
Online AccessGet full text

Cover

More Information
Summary:This article analyzes how Big Data is changing the way we learn from observations. We describe the changes in statistical methods in seven areas that have been shaped by the Big Data-rich environment: the emergence of new sources of information; visualization in high dimensions; multiple testing problems; analysis of heterogeneity; automatic model selection; estimation methods for sparse models; and merging network information with statistical models. Next, we compare the statistical approach with those in computer science and machine learning and argue that the convergence of different methodologies for data analysis will be the core of the new field of data science. Then, we present two examples of Big Data analysis in which several new tools discussed previously are applied, as using network information or combining different sources of data. Finally, the article concludes with some final remarks.
ISSN:1133-0686
1863-8260
DOI:10.1007/s11749-019-00651-9