Definition

Let I(λ)I(\lambda) be a normalized of a , induced from fixed unitary data and a complex character eλe^\lambda of the split factor. A complementary-series representation is an irreducible member or irreducible constituent for a parameter with Reλ0\operatorname{Re}\lambda\neq 0 that nevertheless admits a positive-definite, invariant Hermitian form and hence a unitary Hilbert globalization. It is non-tempered and usually lies in an open parameter region adjacent to the unitary principal series. The term refers to such induced families, not to every non-tempered .

Construction by intertwiners

For a Weyl reflection carrying λ\lambda to λ-\lambda, a suitably normalized can define a GG-invariant Hermitian form on I(λ)I(\lambda). Complementary series occur precisely on those real parameter regions where this form is positive definite; degeneracy often marks reducibility or an endpoint. The invariant form and the distinction between open complementary series and semidefinite endpoint forms are made explicit in Knapp–Stein, §3, pp. 253–257.

Rank-one example

The spherical principal series of SL(2,R)\mathrm{SL}(2,\mathbb R) has a real interval, on either side of the unitary axis and before the first reducibility point, on which the intertwining form is positive. Its irreducible globalizations form the spherical complementary series. At the central imaginary parameter one instead has unitary principal series; beyond the positivity interval the same are not unitarizable.

Conventions and scope
References
  1. Anthony W. Knapp and Elias M. Stein, “The Existence of Complementary Series,” in Problems in Analysis: A Symposium in Honor of Salomon Bochner, Princeton University Press, 1970, 249–259. DOI record. Relevant: §3, especially the invariant Hermitian form on p. 253 and the rank-one positivity result on p. 257.
  2. Anthony W. Knapp, Representation Theory of Semisimple Groups: An Overview Based on Examples, Princeton Mathematical Series 36, Princeton University Press, 1986. Author-maintained record. Relevant: Chapters VII and XVI on principal-series intertwiners, complementary series, and unitarity.