Information Theory
Information theory quantifies information, uncertainty, compression, and transmission through entropy and coding.
The two largest fields, with nearly two thirds of the articles between them, are source coding and error-correcting codes. Source coding is mostly Huffman coding and its optimality, with Shannon's source coding theorem, which puts the least expected length of a uniquely decodable code over \(q \ge 2\) letters between the base-\(q\) entropy and one letter more. Error-correcting codes run from linear codes and the perfect Hamming \((7,4)\) code to Reed–Solomon codes, which meet the Singleton bound, and good locally testable codes. Entropy and mutual information and information inequalities such as Fano's set up the basic quantities, and channel simulation asks how small \(H(Y \mid S)\) can be for an auxiliary variable \(S\), independent of the input \(X\), that determines the output \(Y\) together with \(X\).
Loading dependency explorer…
The dependency explorer needs JavaScript and its graph data.