Entropic measures, Markov information sources and complexity

• Published in: Applied mathematics and computation Vol. 132; no. 2; pp. 369 - 384
• Main Authors: Calude, Cristian S., Dumitrescu, Monica
• Format: Journal Article
• Language: English
• Published: New York, NY Elsevier Inc 10.11.2002; Elsevier
• Subjects: Algorithmic probability; Applied sciences; Computer science; control theory; systems; Entropy rate; Exact sciences and technology; Information systems. Data bases; Markov processes; Mathematics; Memory organisation. Data processing; Probability and statistics; Probability theory and stochastic processes; Program-size complexity; Sciences and techniques of general use; Shannon's entropy; Software; Entropy rate; Shannon's entropy; Program-size complexity; Algorithmic probability; Information source; Algorithmics; Distribution function; Entropy; Probability distribution; Complexity measure; Shannon theory; Communication theory; Information theory

The concept of entropy plays a major part in communication theory. The Shannon entropy is a measure of uncertainty with respect to a priori probability distribution. In algorithmic information theory the information content of a message is measured in terms of the size in bits of the smallest progra...