Information Theory
This theory was from Shannon in a 1948 paper. The theory helps up measure how much we know. Measurements grounded by Information Theory are use to deep learning because these measurements provide us data upon which we can estimate performance.
Situation, we need to evaluate the performance of an experimental deep learning project.