Disturbed-entropy: A simple data quality assessment approach

From the perspective of information value, we proposed a simple and effective approach to assess data quality, called disturbed-entropy. In specific, considering image classification task, the existing samples per category are statistically represented as a pixel prototype, which is used to disturb...

Full description

Saved in:
Bibliographic Details
Published inICT express Vol. 8; no. 3; pp. 309 - 312
Main Authors Li, Yang, Chao, Xuewei, Ercisli, Sezai
Format Journal Article
LanguageEnglish
Published Elsevier 01.09.2022
Subjects
Data-centric computing
Information entropy
Neural network
Online AccessGet full text

Cover

More Information
Summary:From the perspective of information value, we proposed a simple and effective approach to assess data quality, called disturbed-entropy. In specific, considering image classification task, the existing samples per category are statistically represented as a pixel prototype, which is used to disturb the unseen samples. Then, the entropy of disturbed image is calculated based on predicted probability. Both the numerical and visual experiments are conducted to show the effect. In case of same data budget, the performance comparison based on selected good and bad data is significant and consistent. This work attempts to gain insight into data quality and redundancy.
ISSN:2405-9595
2405-9595
DOI:10.1016/j.icte.2022.01.006