# Book:Robert B. Ash/Information Theory

Jump to navigation
Jump to search
## Robert B. Ash:

## Robert B. Ash: *Information Theory*

Published $\text {1965}$, **Dover Publications**

- ISBN 0-486-66521-6

### Contents

- Preface

- Chapter One:
**A Measure of Information**- 1.1 Introduction
- 1.2 Axioms for the Uncertainty Measure
- 1.3 Three Interpretations of the Uncertainty Function
- 1.4 Properties of the Uncertainty Function; Joint and Conditional Uncertainty
- 1.5 The Measure of Information
- 1.6 Notes and Remarks

- Chapter Two:
**Noiseless Coding**- 2.1 Introduction
- 2.2 The Problem of Unique Decipherability
- 2.3 Necessary and Sufficient Conditions for the Existence of Instantaneous Codes
- 2.4 Extension of the Condition $\ds \sum_{i \mathop = 1}^M D^{-n_i} \le 1$ to Uniquely Decipherable Codes
- 2.5 The Noiseless Coding Theorem
- 2.6 Construction of Optimal Codes
- 2.7 Notes and Remarks

- Chapter Three:
**The Discrete Memoryless Channel**- 3.1 Models for Communication Channels
- 3.2 The Information Processed by a Channel; Channel Capacity; Classification of Channels
- 3.3 Calculation of Channel Capacity
- 3.4 Decoding Schemes; the Ideal Observer
- 3.5 The Fundamental Theorem
- 3.6 Exponential Error Bounds
- 3.7 The Weak Converse to the Fundamental Theorem
- 3.8 Notes and Remarks

- Chapter Four:
**Error Correcting Codes**- 4.1 Introduction; Minimum Distance Principle
- 4.2 Relation between Distance and Error Correcting Properties of Codes; the Hamming Bound
- 4.3 Parity Check Coding
- 4.4 The Application of Group Theory to Parity Check Coding
- 4.5 Upper and Lower Bounds on the Error Correcting Ability of Parity Check Codes
- 4.6 Parity Check Codes Are Adequate
- 4.7 Precise Error Bounds for General Binary Codes
- 4.8 The Strong Converse for the Binary Symmetric Channel
- 4.9 Non-Binary Coding
- 4.10 Notes and Remarks

- Chapter Five:
**Further Theory of Error Correcting Codes**- 5.1 Feedback Shift Registers and Cyclic Codes
- 5.2 General Properties of Binary Matrices and Their Cycle Sets
- 5.3 Properties of Cyclic Codes
- 5.4 Bose-Chaudhuri-Hocquenghem Codes
- 5.5 Single Error Correcting Cyclic Codes; Automatic Decoding
- 5.6 Notes and Remarks

- Chapter Six:
**Information Sources**- 6.1 Introduction
- 6.2 A Mathematical Model for an Information Source
- 6.3 Introduction to the Theory of Finite Markov Groups
- 6.4 Information Sources; Uncertainty of a Source
- 6.5 Order of a Source; Approximation of a General Information Source by a Source of Finite Order
- 6.6 The Asymptotic Equipartition Property
- 6.7 Notes and Remarks

- Chapter Seven:
**Channels with Memory**- 7.1 Introduction
- 7.2 The Finite-State Channel
- 7.3 The Coding Theorem for Finite State Regular Channels
- 7.4 The Capacity of a General Discrete Channel; Comparison of the Weak and Strong Converses
- 7.5 Notes and Remarks

- Chapter Eight:
**Continuous Channels**- 8.1 Introduction
- 8.2 The Time-Discrete Gaussian Channel
- 8.3 Uncertainty in the Continuous Case
- 8.4 The Converse to the Coding Theorem for the Time-Discrete Gaussian Channel
- 8.5 The Time-Continuous Gaussian Channel
- 8.6 Band-Limited Channels
- 8.7 Notes and Remarks

**Appendix**- 1. Compact and Symmetric Operators on $L_2 \sqbrk {a, b}$
- 2. Integral Operators
- 3. The Karhunen-Loève Theorem
- 4. Further Results Concerning Integral Operators Determined by a Covariance Function

**Tables of Values of $-\log_2 p$ and $-p \log_2 p$**

**Solutions to Problems**

**References**

**Index**