Jensen's inequality (lemma)

Definitions and notation

• Probability measure and expectation .
• Convex function .

Statement

Let $(\Omega,\mathcal F,\mathbb P)$ be a probability space and let $X:\Omega\to I$ be a random variable taking values in a convex set $I\subseteq \mathbb R^d$. Let $\varphi:I\to\mathbb R$ be convex.

Assume $X$ is integrable and $\varphi(X)$ is integrable, i.e.

• $\mathbb E[\|X\|] < \infty$ and
• $\mathbb E[|\varphi(X)|] < \infty$.

Then

If $\varphi$ is concave, the inequality is reversed:

Key hypotheses and conclusions

Hypotheses

• $X$ is a random variable with values in a convex set $I$.
• $\varphi$ is convex on $I$.
• The expectations $\mathbb E[X]$ and $\mathbb E[\varphi(X)]$ exist (finite).

Conclusions

• Convexity pulls outside expectations: $\varphi(\mathbb E[X]) \le \mathbb E[\varphi(X)]$.
• If $\varphi$ is strictly convex (and mild regularity/“non-degenerate support” assumptions hold), equality forces $X$ to be almost surely constant.

Proof idea / significance

A standard proof uses supporting hyperplanes: convexity implies that at the point $\mathbb E[X]$ there exists a subgradient $g$ such that $\varphi(y) \ge \varphi(\mathbb E[X]) + g\cdot (y-\mathbb E[X])$ for all $y\in I$. Substitute $y=X(\omega)$ and take expectations; the linear term vanishes because $\mathbb E[X-\mathbb E[X]]=0$.

In statistical mechanics, Jensen’s inequality is a basic engine behind entropy/variational bounds, including Gibbs' inequality and exponential-moment bounds such as Chernoff bounds .