Canonical: https://knowlpedia.com/probability/

# Probability Theory

Foundations of probability theory and information theory

## Foundations

- [Probability measure](https://knowlpedia.com/probability/probability-measure/)
- [Probability space](https://knowlpedia.com/probability/probability-space/)
- [Event](https://knowlpedia.com/probability/event-probability/)

## Random Variables

- [Random variable](https://knowlpedia.com/probability/random-variable/)
- [Random vector](https://knowlpedia.com/probability/random-vector/)
- [Distribution (law)](https://knowlpedia.com/probability/distribution-law/)
- [Identically distributed](https://knowlpedia.com/probability/identically-distributed/)
- [IID sequence](https://knowlpedia.com/probability/iid-sequence/)
- [Probability mass function](https://knowlpedia.com/probability/probability-mass-function/)
- [Probability density function](https://knowlpedia.com/probability/probability-density-function/)
- [Cumulative distribution function](https://knowlpedia.com/probability/cumulative-distribution-function/)
- [Bernoulli distribution](https://knowlpedia.com/probability/bernoulli-distribution/)
- [Uniform distribution](https://knowlpedia.com/probability/uniform-distribution/)
- [Normal distribution](https://knowlpedia.com/probability/normal-distribution/)

## Expectation and Moments

- [Expectation](https://knowlpedia.com/probability/expectation/)
- [Expectation of a function](https://knowlpedia.com/probability/expectation-function-of-rv/)
- [Conditional expectation](https://knowlpedia.com/probability/conditional-expectation/)
- [Conditional probability](https://knowlpedia.com/probability/conditional-probability/)
- [Variance](https://knowlpedia.com/probability/variance/)
- [Covariance](https://knowlpedia.com/probability/covariance/)
- [Correlation coefficient](https://knowlpedia.com/probability/correlation-coefficient/)
- [Moment](https://knowlpedia.com/probability/moment/)

## Generating Functions

- [Moment generating function](https://knowlpedia.com/probability/moment-generating-function/)
- [Characteristic function](https://knowlpedia.com/probability/characteristic-function-probability/)
- [Cumulant generating function](https://knowlpedia.com/probability/cumulant-generating-function/)
- [Cumulant](https://knowlpedia.com/probability/cumulant/)

## Independence

- [Independence of events](https://knowlpedia.com/probability/independence-events/)
- [Independence of σ-algebras](https://knowlpedia.com/probability/independence-sigma-algebras/)
- [Independence of random variables](https://knowlpedia.com/probability/independence-random-variables/)

## Concentration Inequalities

- [Markov inequality](https://knowlpedia.com/probability/markov-inequality/)
- [Chebyshev inequality](https://knowlpedia.com/probability/chebyshev-inequality/)
- [Chernoff bound](https://knowlpedia.com/probability/chernoff-bound/)

## Limit Theorems

- [Weak Law of Large Numbers](https://knowlpedia.com/probability/weak-law-large-numbers/)
- [Strong Law of Large Numbers](https://knowlpedia.com/probability/strong-law-large-numbers/)
- [Central Limit Theorem](https://knowlpedia.com/probability/central-limit-theorem/)

## Information Theory

- [Shannon entropy](https://knowlpedia.com/probability/shannon-entropy/)
- [Differential entropy](https://knowlpedia.com/probability/differential-entropy/)
- [Relative entropy (KL divergence)](https://knowlpedia.com/probability/relative-entropy-kl-divergence/)
- [Gibbs inequality](https://knowlpedia.com/probability/gibbs-inequality-kl/)
- [Total variation distance](https://knowlpedia.com/probability/total-variation-distance/)
- [Pinsker inequality](https://knowlpedia.com/probability/pinsker-inequality/)
- [Maximum entropy principle](https://knowlpedia.com/probability/maximum-entropy-principle/)

## Measure-Theoretic Foundations

- [Radon–Nikodym theorem](https://knowlpedia.com/probability/radon-nikodym-theorem/)
- [Gaussian Measure on a Hilbert Space (Segal)](https://knowlpedia.com/probability/gaussian-measure-hilbert-space/)
- [Tame Function (Segal)](https://knowlpedia.com/probability/tame-function/)

## Cones, averages and moment constraints

- [Covariance matrix](https://knowlpedia.com/probability/covariance-matrix/)
- [Derivatives of the log moment generating function](https://knowlpedia.com/probability/derivatives-of-log-mgf/)
- [Exponential tilt](https://knowlpedia.com/probability/exponential-tilt/)
- [Variance of a normalized exponential weight](https://knowlpedia.com/probability/variance-of-normalized-exponential-weight/)

## Connections with ergodic theory

- [Standard Gaussian probability space](https://knowlpedia.com/probability/standard-gaussian-probability-space/)
- [Probability integral transform](https://knowlpedia.com/probability/probability-integral-transform/)

See the [ergodic theory reading paths](https://knowlpedia.com/ergodic-theory/) for the surrounding constructions and theorems.
