Section
Asymptotics
Asymptotic methods and approximations
Stirling's approximation gives accurate large- estimates for the factorial (factorial function) and related quantities such as binomial and multinomial coefficients.
Multinomial coefficients have exponential growth rates governed by entropy. This connection underlies many results in information theory and large deviations, including the method-of-types.
Laplace's method approximates integrals of the form
for large , when the main contribution comes from a neighborhood of the point where is maximal. It is the real-variable analogue of the complex saddle-point-method.
The saddle-point method (also called steepest descent in many contexts) is a complex-analytic technique for approximating integrals like
for large , where is a contour in the complex plane. It is closely related to laplaces-method, but uses complex contours chosen to pass through a saddle point (a stationary point of the phase).
The method of types is a set of counting and probability tools for sequences over a finite alphabet. It is foundational in information theory and large deviations: it turns questions about probabilities of empirical frequencies into entropy and divergence calculations.
It relies on the entropy–counting relationship in entropy-multinomial-coefficients.