Gibbs entropy (Shannon form)

Definition (Gibbs/Shannon entropy).
An equilibrium ensemble is a probability measure on microstates (often given by a density with respect to a reference measure). For a discrete set of microstates with probabilities $\{p_i\}$, the Gibbs (Shannon) entropy is

This is $k_B$ times the Shannon entropy .

For a classical phase-space description, let $d\Gamma$ denote the phase-space volume element and let $\rho(x)$ be a probability density on phase space (so $\int \rho\,d\Gamma=1$, interpreted via the Lebesgue integral ). Then

Remark on “continuous entropy.”
In the continuous setting, $\rho$ is a density with respect to a chosen reference measure (here the Liouville/phase-space volume). Changing that reference shifts $S_G$ by an additive constant. In statistical mechanics, this ambiguity matches the physical fact that entropy is defined up to an additive constant once a coarse-graining (or a microscopic phase-space cell) is fixed.

Canonical example.
In the canonical ensemble , with Hamiltonian $H(x)$, inverse temperature β , and partition function $Z(\beta)$, the density is

Using the ensemble average $U=\langle H\rangle_{\rho_\beta}$, one obtains the identity

This links Gibbs entropy directly to the logarithm of the partition function and to statistical free energy .

Physical interpretation.
$S_G$ measures uncertainty (spread) of the ensemble over microstates: it is large when probability mass is distributed over many microstates and small when concentrated. It is also the entropy functional appearing in entropy maximization constructions such as maximum-entropy construction of thermal states .

Inequalities and irreversibility viewpoint.
Comparing two ensembles $P$ and $Q$ naturally introduces relative entropy (KL divergence) , whose nonnegativity is Gibbs’ inequality . In equilibrium statistical mechanics, this underlies variational characterizations of the canonical state and convexity properties of $\ln Z$.