Total variation distance
A distance between two probability distributions defined by the largest possible difference they assign to the same event.
A total variation distance between probability measures and on the same measurable space is
where the supremum ranges over measurable sets .
If with densities and , then
Examples
- For Bernoulli laws with parameters and , .
- For probability mass functions and on a finite set,