Canonical: https://knowlpedia.com/ergodic-theory/shannon-mcmillan-breiman-theorem/

# Shannon–McMillan–Breiman theorem

Typical orbit-name probabilities decay exponentially at the partition entropy rate.

Let \(T\) be [ergodic](https://knowlpedia.com/ergodic-theory/ergodic-transformation/) and probability preserving, and let \(\mathcal P\) be a [finite measurable partition](https://knowlpedia.com/ergodic-theory/measurable-partition/). With \(\mathcal P^{(n)}=\bigvee_{j=0}^{n-1}T^{-j}\mathcal P\),
\[
-\frac1n\log\mu(\mathcal P^{(n)}(x))\longrightarrow h_\mu(T,\mathcal P)
\quad\text{for almost every }x.
\]
This is the finite-partition **Shannon–McMillan–Breiman theorem**. The union of zero-probability cells over all \(n\) is a null set and can be omitted.

## Interpretation

For a typical trajectory, the probability of observing its length-\(n\) name is \(\exp(-n h_\mu(T,\mathcal P)+o(n))\). If the partition is generating, its [entropy rate](https://knowlpedia.com/ergodic-theory/kolmogorov-sinai-entropy/) is the entropy of the entire system. The result extends the independent-symbol information law to dependent observations in any ergodic system.

## References

1. Karma Dajani, [*Introduction to Ergodic Theory and its Applications to Number Theory*](https://www.staff.science.uu.nl/~kraai101/LectureNotesMM-2.pdf), 2014, Theorem 5.4.2.
