A Unified Matrix Representation for Calculating Entropy Quantities

• Published in: International journal of electrical engineering & education Vol. 43; no. 2; pp. 164 - 172
• Main Author: Al-Sammak, A. J.
• Format: Journal Article
• Language: English
• Published: London, England SAGE Publications 01.04.2006; Manchester University Press; Sage Publications Ltd
• Subjects: Applied sciences; Communication (Thought Transfer); Communication channels; Curriculum subjects: programmes and methods; Data compression; Educational sciences; Engineering Education; Entropy; Errors; Exact sciences and technology; Information Theory; Information, signal and communications theory; Matrix; Probability; Scientific Concepts; Software packages; Studies; Teaching methods; Technical education; Telecommunications and information theory; Undergraduate Students; entropy calculations; information theory; matrix representation; Channel capacity; MATLAB software; Matrix method; Compact design; Implementation; Technical Education; Information theory
• ISSN: 0020-7209; 2050-4578
• DOI: 10.7227/IJEEE.43.2.6

In this paper a unified framework for presenting various entropy quantities needed in information theory is given. This method allows an easy, elegant and compact presentation and lends itself to easy implementation using standard mathematical/engineering software. Examples illustrating the use of this technique are given together with a general MATLAB program for calculating entropies as well as the channel capacity. The technique was preferred over classical representations by undergraduate students taking a first course on digital communications.