Characteristic Functions

A characteristic function is the Fourier transform of a probability distribution.

For a real random variable $X$, its characteristic function is

$$\chi_X(t) = \mathbb E[e^{itX}].$$

It packages the whole probability law into a function of the Fourier variable $t$. For independent sums, characteristic functions multiply. For distributions with moments, derivatives at $t=0$ recover those moments.

Characteristic functions are especially useful because they exist for every probability distribution, even when an ordinary density or a moment-generating function does not.

Helpful background. Review expectation values to average a function of a random variable.

Characteristic functions as Fourier averages

Let $X$ be a real-valued random variable. The characteristic function of $X$ is

$$\chi_X(t) = \mathbb E[e^{itX}], \qquad t\in\mathbb R.$$

For a discrete random variable with values $x_j$ and probabilities $p_j$,

$$\chi_X(t) = \sum_j p_j e^{itx_j}.$$

For a continuous random variable with density $f_X(x)$,

$$\chi_X(t) = \int_{-\infty}^{\infty} e^{itx}f_X(x)\,dx.$$

This is a Fourier transform of the probability density, using the sign convention common in probability theory.

Basic Properties

Every characteristic function satisfies

$$\chi_X(0)=1,$$

because $e^{i0X}=1$.

It also satisfies the bound

$$\lvert\chi_X(t)\rvert\le1,$$

because it is the average of complex numbers of unit magnitude.

If $X$ and $Y$ have the same characteristic function for all real $t$, then they have the same distribution. Thus the characteristic function determines the probability law.

Relation to Fourier Transform

For a density $f_X$, the characteristic function is

$$\chi_X(t) = \int e^{itx}f_X(x)\,dx.$$

Compared with the Fourier convention used on the main Fourier Transform page,

$$\mathcal F[f](k) = \int f(x)e^{-ikx}\,dx,$$

one has

$$\chi_X(t) = \mathcal F[f_X](-t).$$

When the inversion conditions hold, the density can be recovered by

$$f_X(x) = \frac{1}{2\pi} \int_{-\infty}^{\infty} e^{-itx}\chi_X(t)\,dt.$$

If no ordinary density exists, the inversion statement must be understood distributionally or in terms of probability measures.

Moments from Derivatives

If $X$ has finite moments up to order $n$ and the corresponding differentiations are justified, then

$$\chi_X^{(n)}(0) = i^n\mathbb E[X^n].$$

Equivalently,

$$\mathbb E[X^n] = (-i)^n\chi_X^{(n)}(0).$$

In particular,

$$\mathbb E[X] = \frac{1}{i}\chi_X'(0),$$

and

$$\mathbb E[X^2] = -\chi_X''(0).$$

The variance is then

$$\operatorname{Var}(X) = -\chi_X''(0) - \left[ \frac{1}{i}\chi_X'(0) \right]^2.$$

One must be cautious: a characteristic function always exists, but its derivatives at the origin may not encode finite moments if those moments diverge.

Centering and Scaling

If

$$Y=aX+b,$$

then

$$\chi_Y(t) = e^{itb}\chi_X(at).$$

Thus shifting a random variable multiplies the characteristic function by a phase, while scaling changes the Fourier variable.

This is the same structural rule as translation and scaling in Fourier analysis.

Independent Sums

If $X$ and $Y$ are independent, then

$$\chi_{X+Y}(t) = \chi_X(t)\chi_Y(t).$$

Proof:

$$\begin{aligned} \chi_{X+Y}(t) &= \mathbb E[e^{it(X+Y)}]\\ &= \mathbb E[e^{itX}e^{itY}]\\ &= \mathbb E[e^{itX}]\mathbb E[e^{itY}]\\ &= \chi_X(t)\chi_Y(t). \end{aligned}$$

The key step is independence. Without independence, the expectation of the product need not factor.

For independent $X_1,\ldots,X_n$,

$$\chi_{X_1+\cdots+X_n}(t) = \prod_{j=1}^n\chi_{X_j}(t).$$

This product rule is one of the main reasons characteristic functions are central in limit theorems.

Examples

For a point mass at $a$,

$$X=a \quad\text{with probability }1,$$

so

$$\chi_X(t) = e^{ita}.$$

For a two-outcome variable with values $+1$ and $-1$,

$$\mathbb P(X=+1)=q, \qquad \mathbb P(X=-1)=1-q,$$

the characteristic function is

$$\chi_X(t) = qe^{it}+(1-q)e^{-it}.$$

For a normal distribution with mean $\mu$ and variance $s^2$,

$$\chi_X(t) = \exp \left( i\mu t-\frac12s^2t^2 \right).$$

The Gaussian is special because independent Gaussian sums remain Gaussian; the product of their characteristic functions is again a Gaussian characteristic function. The density, covariance, and Gaussian-integral facts are collected in Gaussian Distributions.

Cumulants

The logarithm of the characteristic function,

$$K_X(t) = \log\chi_X(t),$$

is the cumulant-generating function in Fourier form, near $t=0$ when the logarithm is well behaved.

If the logarithm is analytic near zero, it has the convergent expansion

$$K_X(t) = \sum_{n=1}^{\infty} \kappa_n \frac{(it)^n}{n!},$$

where $\kappa_n$ is the $n$th cumulant. Finite moments through order $n$ suffice for derivatives and a finite Taylor expansion through that order; existence of all moments alone does not guarantee convergence of the infinite series. The first two are

$$\kappa_1=\mathbb E[X], \qquad \kappa_2=\operatorname{Var}(X).$$

For independent variables, cumulants add because characteristic functions multiply and logarithms turn products into sums.

Central-Limit Intuition

Let $X_1,X_2,\ldots$ be independent and identically distributed with

$$\mathbb E[X_j]=0, \qquad \operatorname{Var}(X_j)=1.$$

For small $u$, the characteristic function has the expansion

$$\chi_X(u) = 1-\frac{u^2}{2}+o(u^2),$$

assuming the variance exists.

For the normalized sum

$$S_n = \frac{X_1+\cdots+X_n}{\sqrt n},$$

independence gives

$$\chi_{S_n}(t) = \left[ \chi_X \left( \frac{t}{\sqrt n} \right) \right]^n.$$

Using the small-$u$ expansion,

$$\chi_{S_n}(t) \approx \left( 1-\frac{t^2}{2n} \right)^n \longrightarrow e^{-t^2/2}.$$

The limiting characteristic function $e^{-t^2/2}$ is that of a standard normal distribution. This is the characteristic-function intuition behind the central limit theorem.

Quantum Observables

For a quantum observable $A$ in a state $\rho$, the outcome distribution has a characteristic function

$$\chi_A(t) = \operatorname{Tr}(\rho e^{itA}).$$

For a pure state,

$$\chi_A(t) = \langle\psi\vert e^{itA}\lvert\psi\rangle.$$

This is just the characteristic function of the Born-rule probability distribution for measuring $A$. The operator exponential is defined through the spectral theorem, reviewed in Spectral Theorem, Practical Version.

Expanding around $t=0$ gives

$$\chi_A(t) = 1+it\langle A\rangle - \frac{t^2}{2}\langle A^2\rangle +\cdots,$$

as a finite expansion to the order justified by the finite moments. For example, a finite second moment gives a remainder $o(t^2)$ after the quadratic term. No norm-convergent operator Taylor series for an unbounded $A$ is being assumed.

This should not be confused with more specialized phase-space characteristic functions used for Wigner functions and continuous-variable quantum information. Those are related but carry additional structure.

Common Mistakes

• Forgetting the sign convention difference between probability characteristic functions and some Fourier-transform tables.
• Assuming a characteristic function requires an ordinary probability density.
• Differentiating at $t=0$ without checking that the corresponding moment exists.
• Multiplying characteristic functions for sums without independence.
• Confusing the characteristic function of a measurement-outcome distribution with the wavefunction itself.
• Treating central-limit behavior as automatic when variance is infinite or independence assumptions fail.

References

• P. Billingsley, Probability and Measure, 3rd ed., Wiley, 1995.
• R. Durrett, Probability: Theory and Examples, 5th ed., Cambridge University Press, 2019 — probability measures, moments, and limit theorems.
• W. Feller, An Introduction to Probability Theory and Its Applications, Volume II, 2nd ed., Wiley, 1971.
• G. B. Folland, Fourier Analysis and Its Applications, American Mathematical Society, 2009.
• B. C. Hall, Quantum Theory for Mathematicians, Springer, 2013.

Exercises

1. Let $X$ take values $0$ and $1$ with $\mathbb P(X=1)=q$. Find $\chi_X(t)$.

Solution

The probabilities are $\mathbb P(X=0)=1-q$ and $\mathbb P(X=1)=q$. Therefore

$$\chi_X(t) = (1-q)e^{it0}+qe^{it} = 1-q+qe^{it}.$$

2. Use the previous answer to compute $\mathbb E[X]$ from $\chi_X'(0)$.

Solution

Differentiate:

$$\chi_X'(t) = iqe^{it}.$$

Thus

$$\chi_X'(0)=iq.$$

Since $\mathbb E[X]=\chi_X'(0)/i$, one gets

$$\mathbb E[X]=q.$$

3. If $X$ and $Y$ are independent standard normal variables, use characteristic functions to identify the distribution of $X+Y$.

Solution

For a standard normal variable,

$$\chi_X(t)=e^{-t^2/2}.$$

Independence gives

$$\chi_{X+Y}(t) = e^{-t^2/2}e^{-t^2/2} = e^{-t^2}.$$

This has the form $\exp(-s^2t^2/2)$ with $s^2=2$. Thus $X+Y$ is normal with mean $0$ and variance $2$.

4. Show that if $Y=aX+b$, then $\chi_Y(t)=e^{itb}\chi_X(at)$.

Solution

Use the definition:

$$\chi_Y(t) = \mathbb E[e^{it(aX+b)}] = e^{itb}\mathbb E[e^{i(at)X}] = e^{itb}\chi_X(at).$$

5. Let $A$ be self-adjoint and $\psi$ normalized with finite spectral second moment. Show that $\chi_A(t)=1+it\langle A\rangle+O(t^2)$.

Solution

Use the scalar inequality $|e^{iu}-1-iu|\le u^2/2$ inside the spectral integral. Its remainder is at most $t^2\int a^2\,d\mu_\psi(a)/2$. Consequently,

$$\chi_A(t) = \langle\psi\vert e^{itA}\lvert\psi\rangle = 1+it\langle\psi\vert A\lvert\psi\rangle+O(t^2).$$

The coefficient of $it$ is therefore $\langle A\rangle$.