Information Theory and Reliable Communication : 9780471290483

This book is designed primarily for use as a first-year graduate text in information theory, suitable for both engineers and mathematicians. It is assumed that the reader has some understanding of freshman calculus and elementary probability, and in the later chapters some introductory random process theory. Unfortunately there is one more requirement that is harder to meet. The reader must have a reasonable level of mathematical maturity and capability for abstract thought. The major results of the theory are quite subtle and abstract and must sometimes be arrived at by what appears to be rather devious routes. Fortunately, recent simplifications in the theory have made the major results more accessible than in the past.

Because of the subtlety and abstractness of the subject, it is necessary to be more rigorous than is usual in engineering. I have attempted to soften this wherever possible by preceding the proof of difficult theorems both with some explanation of why the theorem js important and with an intuitive explanation of why it is true. An attempt has also been made to provide the simplest and most elementary proof of each theorem, and many of the proofs here are new. l have carefully avoided the rather obnoxious practice in many elementary textbooks of quoting obscure mathematical theorems in the middle of a proof to make it come out right.