## Claude Shannon

01 Mar 2001

**Claude Shannon died on Tuesday,** at 86. Shannon was the father of information theory, and a student of Vannevar Bush, a thinker whose work in the 1940s laid the foundations for what we know as the web. As part of his work in information theory, Shannon introduced the term ‘bit’. And by the end of his life, just a few decades later, we have a network transmitting multiple petabytes every day. From his 1948 paper *A Mathematical Theory of Communication*:

The ratio of the entropy of a source to the maximum value it could have while still restricted to the same

symbols will be called its relative entropy. This is the maximum compression possible when we encode into

the same alphabet. One minus the relative entropy is the redundancy. The redundancy of ordinary English,

not considering statistical structure over greater distances than about eight letters, is roughly 50%. This

means that when we write English half of what we write is determined by the structure of the language and

half is chosen freely.