Quantum relative entropy

Let $\rho$ and $\sigma$ be density-operators on a finite-dimensional Hilbert space $H$. The quantum relative entropy (also called Umegaki relative entropy) is defined by

The support condition is needed because $\log\sigma$ is not finite on the kernel of $\sigma$. The operators $\log\rho$ and $\log\sigma$ are defined via spectral calculus (see spectrum-self-adjoint-finite).

Classical (commuting) case

If $\rho$ and $\sigma$ commute, they are simultaneously diagonalizable and the formula reduces to the classical relative entropy (Kullback-Leibler divergence) of their eigenvalue distributions; compare relative-entropy-kl-divergence.

Key properties

• Nonnegativity (Klein inequality): $D(\rho\|\sigma)\ge 0$, with equality iff $\rho=\sigma$.
• Not symmetric: typically $D(\rho\|\sigma)\ne D(\sigma\|\rho)$; it is not a metric.
• Data processing inequality: for any completely positive trace-preserving map $\Phi$,
  $$D(\rho\|\sigma)\ \ge\ D(\Phi(\rho)\|\Phi(\sigma)).$$
  In particular, taking $\Phi=\operatorname{Tr}_B$ (see partial-trace) gives
  $$D(\rho_{AB}\|\sigma_{AB})\ \ge\ D(\rho_A\|\sigma_A), \quad \rho_A=\operatorname{Tr}_B(\rho_{AB}),\ \sigma_A=\operatorname{Tr}_B(\sigma_{AB}).$$

Relation to von Neumann entropy

If $d=\dim H$ and $\tau=I/d$ is the maximally mixed state, then

where $S(\rho)$ is the von-neumann-entropy.