Entropic Statistics: Concept, Estimation, and Application in Machine Learning and Knowledge Extraction

The demands for machine learning and knowledge extraction methods have been booming due to the unprecedented surge in data volume and data quality. Nevertheless, challenges arise amid the emerging data complexity as significant chunks of information and knowledge lie within the non-ordinal realm of...

Full description

Saved in:
Bibliographic Details
Published inMachine learning and knowledge extraction Vol. 4; no. 4; pp. 865 - 887
Main Author Zhang, Jialin
Format Journal Article
LanguageEnglish
Published Basel MDPI AG 01.12.2022
Subjects
Algorithms
Bias
Datasets
discrete data
entropic statistics
Entropy
Entropy (Information theory)
Genes
Hypothesis testing
Information theory
information-theoretic quantity
Knowledge management
Machine learning
Methods
non-ordinal data
non-parametric estimation
Nonparametric statistics
Probability distribution
Random variables
Regression analysis
Statistical methods
Statistics
Variance analysis
United States
Online AccessGet full text

Cover

More Information
Summary:The demands for machine learning and knowledge extraction methods have been booming due to the unprecedented surge in data volume and data quality. Nevertheless, challenges arise amid the emerging data complexity as significant chunks of information and knowledge lie within the non-ordinal realm of data. To address the challenges, researchers developed considerable machine learning and knowledge extraction methods regarding various domain-specific challenges. To characterize and extract information from non-ordinal data, all the developed methods pointed to the subject of Information Theory, established following Shannon’s landmark paper in 1948. This article reviews recent developments in entropic statistics, including estimation of Shannon’s entropy and its functionals (such as mutual information and Kullback–Leibler divergence), concepts of entropic basis, generalized Shannon’s entropy (and its functionals), and their estimations and potential applications in machine learning and knowledge extraction. With the knowledge of recent development in entropic statistics, researchers can customize existing machine learning and knowledge extraction methods for better performance or develop new approaches to address emerging domain-specific challenges.
Bibliography:ObjectType-Article-1
SourceType-Scholarly Journals-1
ObjectType-Feature-2
content type line 14
ISSN:2504-4990
2504-4990
DOI:10.3390/make4040044