Pinsker's inequality

Pinsker’s inequality: Let $P$ and $Q$ be probability measures on the same $(\Omega,\mathcal F)$. Then their total variation distance satisfies

where $D(P\|Q)$ is the Kullback–Leibler divergence computed with natural logarithms (if another log base is used, the constant changes accordingly). The inequality is understood to hold trivially when $D(P\|Q)=+\infty$.

Pinsker’s inequality formalizes that small relative entropy forces two laws to be close in a strong, event-wise sense. Together with Gibbs' inequality , it highlights KL divergence as a nonnegative discrepancy measure that quantitatively controls $d_{\mathrm{TV}}$.