Capacity of Finite State Channels Based on Lyapunov Exponents of Random Matrices

• Published in: IEEE transactions on information theory Vol. 52; no. 8; pp. 3509 - 3532
• Main Authors: Holliday, T., Goldsmith, A., Glynn, P.
• Format: Journal Article
• Language: English
• Published: New York, NY IEEE 01.08.2006; Institute of Electrical and Electronics Engineers; The Institute of Electrical and Electronics Engineers, Inc. (IEEE)
• Subjects: Applied sciences; Channel capacity; Channels; Communication channels; Computation; Computational modeling; Convergence; Entropy; Exact sciences and technology; Filters; Finite-state channel; hidden Markov model; Hidden Markov models; Information dissemination; Intersymbol interference; Lyapunov exponent; Lyapunov exponents; Markov analysis; Markov chains; Mathematical analysis; Mathematical models; Matrix; Mutual information; random matrices; Shannon capacity; Space stations; Systems, networks and services of telecommunications; Telecommunications; Telecommunications and information theory; Theorems; Time-varying channels; Transmission and modulation (techniques and equipments); Time variable channel; Channel capacity; Random matrix; Probabilistic approach; Error estimation; hidden Markov model; random matrices; Matrix product; Finite-state channel; Entropy; Continuous function; Markov chain; Central limit theorem; Intersymbol interference; Hidden Markov models; Lyapunov exponent; Numerical simulation; Shannon capacity; Computing method; Mutual information
• ISSN: 0018-9448; 1557-9654
• DOI: 10.1109/TIT.2006.878230

The finite-state Markov channel (FSMC) is a time-varying channel having states that are characterized by a finite-state Markov chain. These channels have infinite memory, which complicates their capacity analysis. We develop a new method to characterize the capacity of these channels based on Lyapunov exponents. Specifically, we show that the input, output, and conditional entropies for this channel are equivalent to the largest Lyapunov exponents for a particular class of random matrix products. We then show that the Lyapunov exponents can be expressed as expectations with respect to the stationary distributions of a class of continuous-state space Markov chains. This class of Markov chains, which is closely related to the prediction filter in hidden Markov models, is shown to be nonirreducible. Hence, much of the standard theory for continuous state-space Markov chains cannot be applied to establish the existence and uniqueness of stationary distributions, nor do we have direct access to a central limit theorem (CLT). In order to address these shortcomings, we utilize several results from the theory of random matrix products and Lyapunov exponents. The stationary distributions for this class of Markov chains are shown to be unique and continuous functions of the input symbol probabilities, provided that the input sequence has finite memory. These properties allow us to express mutual information and channel capacity in terms of Lyapunov exponents. We then leverage this connection between entropy and Lyapunov exponents to develop a rigorous theory for computing or approximating entropy and mutual information for finite-state channels with dependent inputs. We develop a method for directly computing entropy of finite-state channels that does not rely on simulation and establish its convergence. We also obtain a new asymptotically tight lower bound for entropy based on norms of random matrix products. In addition, we prove a new functional CLT for sample entropy and apply this theorem to characterize the error in simulated estimates of entropy. Finally, we present numerical examples of mutual information computation for intersymbol interference (ISI) channels and observe the capacity benefits of adding memory to the input sequence for such channels