Entropy Overview

The von Neumann entropy of a density operator is

$$S(\rho) = - \operatorname{Tr} \bigl( \rho\log\rho \bigr).$$

It is the Shannon entropy of the eigenvalue distribution of $\rho$. It is zero exactly for pure states and increases under uniform mixing of the eigenvalues. It is not a distance between states and does not assign a preferred physical preparation to a density operator.

That mathematical statement supports several distinct uses. Entropy can quantify local mixedness, preparation uncertainty, thermodynamic entropy, source-compression rate, or pure-state bipartite entanglement, but only after the relevant physical structure has been specified. A positive value by itself does not identify why a state is mixed.

Helpful background. Density operators supplies the spectrum of a state, and Shannon entropy explains entropy of a probability distribution. Unless written as $\ln$, logarithms below have base two and entropy is in bits. We assume finite dimensions until the final qualification.

Spectral Definition

Let $\rho$ be a density operator on a $d$-dimensional Hilbert space. Its spectral decomposition is

$$\begin{aligned} \rho &= \sum_{k=1}^{d} p_k \lvert k\rangle\langle k\rvert, \\ p_k&\ge0, \qquad \sum_{k=1}^{d}p_k=1. \end{aligned}$$

The operator logarithm is defined by functional calculus on the support of $\rho$:

$$\log\rho = \sum_{p_k>0} (\log p_k) \lvert k\rangle\langle k\rvert.$$

Although $\log\rho$ diverges on the kernel of $\rho$, the product $\rho\log\rho$ has a continuous extension there because

$$\lim_{x\to0^+}x\log x=0.$$

Thus the convention $0\log0=0$ gives

$$\begin{aligned} S(\rho) &= - \operatorname{Tr} \bigl( \rho\log\rho \bigr) \\ &= - \sum_{k=1}^{d} p_k\log p_k. \end{aligned}$$

The logarithm base fixes the unit:

• $\log_2$ gives bits;
• $\ln$ gives nats;
• multiplying the natural-logarithm convention by Boltzmann’s constant $k_{\mathrm B}$ gives thermodynamic entropy units.

Changing to another base $b>1$ rescales every entropy by a constant:

$$\log_b x = \frac{\ln x}{\ln b}.$$

What the Entropy Depends On

Entropy depends only on the multiset of eigenvalues. If $U$ is unitary, then $U\rho U^\dagger$ has the same spectrum as $\rho$, so

$$S(U\rho U^\dagger) = S(\rho).$$

The eigenvectors may rotate while the entropy remains unchanged. This makes $S$ a basis-independent state function, unlike the Shannon entropy of the outcomes of one chosen measurement.

The formula also explains what entropy does not retain. It does not identify:

• which eigenvector carries which physical label;
• which ensemble preparation produced $\rho$;
• whether the mixedness came from classical randomization, entanglement, noise, or coarse graining;
• which observable should be measured;
• how close two states are when they have the same spectrum but different eigenvectors.

Entropy compresses the spectrum to one number. In dimensions at least three, different spectra can even have the same entropy. For a qubit it determines the two eigenvalues but not their eigenvectors, so it is still not a complete description of the state.

Bounds and Equality Cases

If $r=\operatorname{rank}\rho$, then

$$0 \le S(\rho) \le \log r \le \log d.$$

The lower bound is saturated exactly by pure states. A pure density operator has spectrum

$$(1,0,\ldots,0),$$

and therefore

$$S(\rho_\psi)=0.$$

Conversely, every term $-p_k\log p_k$ is nonnegative, and it vanishes only when $p_k$ is $0$ or $1$. Normalization then implies that zero entropy requires one eigenvalue equal to $1$, hence a rank-one state. In finite dimensions,

$$\begin{aligned} \rho\ \text{is pure} &\Longleftrightarrow \rho^2=\rho \\ &\Longleftrightarrow \operatorname{Tr}(\rho^2)=1 \\ &\Longleftrightarrow S(\rho)=0. \end{aligned}$$

The upper bound $\log r$ is saturated when the state is uniform on its support:

$$\rho_{\mathrm{supp}} = \frac{1}{r} \Pi_{\mathrm{supp}},$$

where $\Pi_{\mathrm{supp}}$ projects onto an $r$-dimensional subspace. In particular, the unique maximum-entropy state on the full $d$-dimensional space is

$$\rho_* = \frac{I}{d}, \qquad S(\rho_*)=\log d.$$

These are statements at fixed Hilbert-space dimension. Appending unused zero-eigenvalue directions changes $d$ but not $\rho$ on its support and not $S(\rho)$.

Qubit Entropy and the Bloch Radius

A qubit density operator has Bloch form

$$\begin{aligned} \rho_{\boldsymbol r} &= \frac12 \bigl( I+\boldsymbol r\mathbin{\cdot}\boldsymbol\sigma \bigr), \\ r&=\lVert\boldsymbol r\rVert, \qquad 0\le r\le1. \end{aligned}$$

Its eigenvalues are

$$\lambda_\pm = \frac{1\pm r}{2}.$$

Using base-$2$ logarithms,

$$\begin{aligned} S(\rho_{\boldsymbol r}) &= h_2 \left( \frac{1+r}{2} \right), \\ h_2(p) &= -p\log_2p -(1-p)\log_2(1-p). \end{aligned}$$

The direction of $\boldsymbol r$ fixes the eigenvectors but does not affect the entropy. Only the radius matters. For $0<r<1$,

$$\frac{dS}{dr} = \frac12 \log_2 \left( \frac{1-r}{1+r} \right) <0,$$

so entropy decreases monotonically as the Bloch vector approaches the pure-state sphere.

For a qubit, $S(\rho_{\boldsymbol r})=h_2((1+r)/2)$ in bits. Entropy is maximal at the center of the Bloch ball and vanishes on its surface.

For a matrix

$$\rho = \begin{pmatrix} a & c \\ c^* & 1-a \end{pmatrix},$$

the Bloch radius is

$$r = \sqrt{ (2a-1)^2+4\lvert c\rvert^2 }.$$

This yields the entropy without separately solving a quadratic eigenvalue equation. Positivity is equivalent to $0\le r\le1$.

State Entropy Is Not Measurement Entropy

Consider the two qubit states

$$\begin{aligned} \rho_+ &= \lvert+\rangle\langle+\rvert = \frac12 \begin{pmatrix} 1&1\\ 1&1 \end{pmatrix}, \\ \rho_* &= \frac I2 = \frac12 \begin{pmatrix} 1&0\\ 0&1 \end{pmatrix}. \end{aligned}$$

Both have diagonal entries $(1/2,1/2)$ in the computational basis. Measuring either state in that basis gives a fair random bit. Nevertheless,

$$S(\rho_+)=0, \qquad S(\rho_*)=1\ \text{bit}.$$

The off-diagonal coherence makes $\rho_+$ rank one, while $\rho_*$ has two equal eigenvalues. Applying $-x\log x$ entry by entry would miss this distinction and is not how an operator function is evaluated.

More generally, a rank-one projective measurement in basis $\{\lvert i\rangle\}$ has outcome probabilities

$$q_i = \langle i\rvert\rho\lvert i\rangle.$$

Their Shannon entropy satisfies

$$H(\{q_i\}) \ge S(\rho).$$

This can be proved using only the concavity of $f(x)=-x\log_2x$, with $f(0)=0$. If $\rho=\sum_k p_k|k\rangle\langle k|$, set $B_{ik}=|\langle i|k\rangle|^2$. Completeness of both orthonormal bases gives $\sum_k B_{ik}=\sum_i B_{ik}=1$, while $q_i=\sum_k B_{ik}p_k$. Hence

$$H(\{q_i\})=\sum_i f\!\left(\sum_k B_{ik}p_k\right) \ge\sum_{i,k}B_{ik}f(p_k) =\sum_k f(p_k)=S(\rho).$$

Strict concavity makes equality possible only when each measurement ket has support within a single eigenspace of $\rho$. Thus equality holds exactly when $\rho$ is diagonal in the measurement basis, including rotations within degenerate eigenspaces. Measuring in an eigenbasis realizes the minimum: the state entropy is the least outcome entropy among complete rank-one projective measurements.

The restriction on measurements matters. A one-outcome POVM $\{I\}$ has zero outcome entropy for every state and need not obey the inequality. A measurement basis that does not diagonalize $\rho$ adds outcome randomness beyond its spectral mixedness.

Entropy and Ensemble Information

Suppose a preparation procedure uses pure states $\lvert\psi_j\rangle$ with probabilities $q_j$:

$$\rho = \sum_j q_j \lvert\psi_j\rangle\langle\psi_j\rvert.$$

The Shannon entropy $H(\{q_j\})$ describes uncertainty in the classical preparation label $j$. The von Neumann entropy $S(\rho)$ describes the spectrum of the state after that label is discarded. They are generally different because the vectors $\lvert\psi_j\rangle$ need not be distinguishable.

For a pure-state ensemble,

$$S(\rho) \le H(\{q_j\}).$$

If the nonzero-probability states are mutually orthogonal, the preparation labels can be perfectly distinguished and equality holds. At the opposite extreme, if every $\lvert\psi_j\rangle$ is the same vector, then $\rho$ is pure and

$$S(\rho)=0$$

even if $H(\{q_j\})$ is large. The density operator forgets redundant classical labels.

This distinction is essential because a density operator has many ensemble decompositions. Entropy is the same for all of them. See Ensembles and Preparation Procedures and Classical Mixtures vs Quantum Superpositions.

Elementary Structural Properties

Three properties recur throughout quantum mechanics.

Unitary invariance

Unitary evolution preserves the spectrum:

$$S(U\rho U^\dagger) = S(\rho).$$

Therefore a closed system evolving unitarily has constant von Neumann entropy. Apparent entropy change requires a changed description, such as discarding a subsystem, averaging over uncertain controls, performing a nonunitary update, or coupling to an environment and ignoring part of it.

Additivity for product states

If $\rho_A$ has eigenvalues $\{p_i\}$ and $\sigma_B$ has eigenvalues $\{q_j\}$, then $\rho_A\otimes\sigma_B$ has eigenvalues $\{p_iq_j\}$. Hence

$$\begin{aligned} S(\rho_A\otimes\sigma_B) &= - \sum_{i,j} p_iq_j \log(p_iq_j) \\ &= - \sum_i p_i\log p_i {}- \sum_j q_j\log q_j \\ &= S(\rho_A)+S(\sigma_B). \end{aligned}$$

Additivity here assumes a product state. Correlations alter the joint entropy and are quantified by quantities such as Mutual Information.

Concavity under mixing

For density operators $\rho_j$ and classical probabilities $q_j$,

$$S \left( \sum_j q_j\rho_j \right) \ge \sum_j q_j S(\rho_j).$$

Forgetting which preparation occurred cannot lower the average entropy. This does not mean that entropy increases under every quantum channel. A nonunital process such as cooling or amplitude damping can reduce a system’s entropy by transferring entropy to an environment.

In finite dimension, entropy is also continuous: states close in trace distance have close entropies, with a dimension-dependent bound. The Fannes–Audenaert inequality makes this statement quantitative. Continuity becomes subtler in infinite-dimensional systems.

Relation to Purity and Rényi Entropies

The purity

$$\gamma = \operatorname{Tr}(\rho^2) = \sum_k p_k^2$$

and the von Neumann entropy are both spectral mixedness diagnostics, but they retain different information. The order-$2$ Rényi entropy is

$$S_2(\rho) = - \log \operatorname{Tr}(\rho^2),$$

whereas von Neumann entropy is the order-$1$ limit of the Rényi family. With the same logarithm base,

$$S(\rho)\ge S_2(\rho).$$

For a qubit, purity determines the Bloch radius through

$$\operatorname{Tr}(\rho^2) = \frac{1+r^2}{2},$$

so it also determines $S(\rho)$. In dimensions $d\ge3$, equal purity does not generally imply equal von Neumann entropy. They should not be interchanged: purity equals one for a pure state, whereas its von Neumann entropy is zero.

Entanglement Entropy Preview

Let $\lvert\Psi\rangle_{AB}$ be a pure bipartite state with Schmidt decomposition

$$\lvert\Psi\rangle_{AB} = \sum_k \sqrt{\lambda_k}\, \lvert a_k\rangle_A \lvert b_k\rangle_B.$$

The reduced states have the same nonzero spectrum:

$$\begin{aligned} \rho_A &= \sum_k \lambda_k \lvert a_k\rangle\langle a_k\rvert, \\ \rho_B &= \sum_k \lambda_k \lvert b_k\rangle\langle b_k\rvert. \end{aligned}$$

Therefore

$$S(\rho_A) = S(\rho_B) = - \sum_k \lambda_k\log\lambda_k.$$

For a pure joint state, this common value is the entanglement entropy across the $A$ versus $B$ split. It vanishes exactly for a product state. For the Bell state,

$$\lvert\Phi^+\rangle = \frac{ \lvert00\rangle+\lvert11\rangle }{ \sqrt2 },$$

the joint state has zero entropy while each qubit has entropy one bit:

$$S(\rho_{AB})=0, \qquad S(\rho_A)=S(\rho_B)=1.$$

There is no contradiction: entropy is assigned to a specified state on a specified system. A pure whole can have mixed parts.

The qualifier pure joint state is indispensable. If $\rho_{AB}$ is mixed, then $S(\rho_A)$ can reflect local noise, classical preparation uncertainty, environmental entanglement, and several kinds of correlation. It is not by itself a mixed-state entanglement measure.

Thermal and Open-System Interpretations

For a Gibbs state with finite nonzero partition function and finite mean energy

$$\rho_\beta = \frac{e^{-\beta H}}{Z}, \qquad Z=\operatorname{Tr}(e^{-\beta H}),$$

the natural-logarithm entropy is

$$\begin{aligned} S(\rho_\beta) &= - \operatorname{Tr} \bigl( \rho_\beta\ln\rho_\beta \bigr) \\ &= \beta\langle H\rangle_\beta + \ln Z. \end{aligned}$$

Here $\beta=1/(k_{\mathrm B}T)$, and this displayed entropy uses natural logarithms, so it is in nats. Multiplication by $k_{\mathrm B}$ gives the thermodynamic entropy. This interpretation uses the Hamiltonian, equilibrium ensemble, and temperature; an arbitrary mixed state is not automatically thermal. The detailed treatment is Entropy in Quantum Statistical Mechanics.

For a subsystem coupled to an environment, the reduced entropy may change even while the total state evolves unitarily and the total entropy remains constant. Correlations can move information out of the subsystem description. Yet local entropy can also decrease, as in cooling, measurement conditioned on an outcome, or transfer of a mixed state into the environment. Entropy increase is not a universal statement about every reduced quantum process.

Infinite-Dimensional and QFT Caveats

For a trace-class density operator on a separable Hilbert space, the spectral series

$$S(\rho) = - \sum_k p_k\log p_k$$

still defines the von Neumann entropy, but the sum may diverge to $+\infty$. Entropy is not uniformly continuous on an unrestricted infinite-dimensional state space; energy constraints or other compactness conditions are often needed for stable bounds.

In quantum field theory, spatial regions carry infinitely many short-distance degrees of freedom. Naive entanglement entropy is typically regulator dependent and ultraviolet divergent, and algebraic local states need not be represented by density matrices on a simple tensor factor. Finite-dimensional formulas remain useful guides, but their assumptions must be checked before extrapolation.

Calculation Workflow

To compute $S(\rho)$ in finite dimension:

1. Verify that $\rho$ is Hermitian, positive semidefinite, and trace one.
2. Find its eigenvalues, including multiplicities.
3. Discard no small eigenvalue merely because it is inconvenient; distinguish numerical noise from physical support.
4. Choose and state the logarithm base.
5. Evaluate $-\sum_k p_k\log p_k$ with $0\log0=0$.
6. Check the bound $0\le S(\rho)\le\log\operatorname{rank}\rho$.
7. Interpret the number only after identifying whether $\rho$ describes a whole system, a subsystem, a thermal state, or an ensemble average.

For numerical spectra near zero, evaluate $-p\log p$ with stable special functions or explicit zero handling. Forming a matrix logarithm and then multiplying can be less stable than working directly with eigenvalues.

Common Mistakes

Taking the logarithm of each matrix entry. Entropy is a function of the eigenvalues. The coherent $|+\rangle$ state and $I/2$ have identical computational diagonals but different spectra.

Calling every local entropy entanglement. Reduced entropy quantifies bipartite entanglement when the full state is pure. Mixed-state local entropy alone cannot distinguish classical noise from entanglement.

Confusing outcome entropy with state entropy. Measurement randomness depends on the chosen effects. The bound above concerns complete rank-one projective measurements, not arbitrary POVMs or coarse readouts.

Assuming every physical process increases entropy. A reset can reduce system entropy. For a closed system under unitary evolution, the full density-operator spectrum and entropy remain constant.

References

• K. M. R. Audenaert, “A sharp continuity estimate for the von Neumann entropy,” Journal of Physics A: Mathematical and Theoretical 40, 8127–8136 (2007). DOI: 10.1088/1751-8113/40/28/S18.
• M. A. Nielsen and I. L. Chuang, Quantum Computation and Quantum Information, 10th anniversary edition, Cambridge University Press, 2010, Chapters 2 and 11. Cambridge DOI.
• J. Watrous, The Theory of Quantum Information, Cambridge University Press, 2018, Chapter 5. Author’s book page and manuscript.
• A. Wehrl, “General properties of entropy,” Reviews of Modern Physics 50, 221–260 (1978). DOI: 10.1103/RevModPhys.50.221.

Exercises

1. Let $\rho$ have nonzero eigenvalues $\{p_1,\ldots,p_r\}$. Show that $S(\rho)=0$ if and only if $\rho$ is pure.

Solution

For $0\le p\le1$,

$$-p\log p\ge0,$$

with equality only at $p=0$ or $p=1$. Therefore

$$S(\rho) = - \sum_{k=1}^{r} p_k\log p_k$$

vanishes only if every eigenvalue is $0$ or $1$. Since the eigenvalues sum to one, exactly one is $1$ and all others are zero. The density operator then has rank one and is pure. Conversely, the spectrum of a pure state is $(1,0,\ldots,0)$, which gives zero entropy.

2. Show that the maximally mixed state on a $d$-dimensional space has entropy $\log d$. Why does this saturate the upper bound?

Solution

The state $\rho_*=I/d$ has $d$ eigenvalues equal to $1/d$. Hence

$$\begin{aligned} S(\rho_*) &= - d \left( \frac1d\log\frac1d \right) \\ &= \log d. \end{aligned}$$

The eigenvalues form the uniform distribution. Shannon entropy is maximized by the uniform distribution on a fixed number of outcomes, so no $d$-dimensional density operator can have larger entropy.

3. Derive the qubit formula

$$S(\rho_{\boldsymbol r}) = h_2 \left( \frac{1+r}{2} \right)$$

and determine its endpoint values.

Solution

The operator $\boldsymbol r\mathbin{\cdot}\boldsymbol\sigma$ has eigenvalues $\pm r$, so

$$\lambda_\pm = \frac{1\pm r}{2}.$$

Therefore, in bits,

$$\begin{aligned} S(\rho_{\boldsymbol r}) &= - \lambda_+\log_2\lambda_+ - \lambda_-\log_2\lambda_- \\ &= h_2 \left( \frac{1+r}{2} \right). \end{aligned}$$

At $r=0$, both eigenvalues are $1/2$, giving one bit. At $r=1$, the eigenvalues are $1$ and $0$, giving zero. The result depends on the radius but not the Bloch-vector direction.

4. The states $\rho_+=\lvert+\rangle\langle+\rvert$ and $\rho_*=I/2$ have the same computational-basis diagonal. Compute their state entropies and the Shannon entropies of computational-basis measurement outcomes.

Solution

The spectrum of $\rho_+$ is $(1,0)$, so

$$S(\rho_+)=0.$$

The spectrum of $\rho_*$ is $(1/2,1/2)$, so

$$S(\rho_*)=1\ \text{bit}.$$

Both states give computational-basis probabilities $(1/2,1/2)$ and therefore one bit of measurement-outcome entropy. For $\rho_+$ that randomness comes from measuring in a basis that does not contain the state vector; it is not spectral mixedness.

5. Prove additivity for a product state directly from its eigenvalues.

Solution

Let the eigenvalues of $\rho_A$ and $\sigma_B$ be $p_i$ and $q_j$. The product-state eigenvalues are $p_iq_j$. Then

$$\begin{aligned} S(\rho_A\otimes\sigma_B) &= - \sum_{i,j} p_iq_j \log(p_iq_j) \\ &= - \sum_{i,j} p_iq_j {}\times \bigl( \log p_i+\log q_j \bigr) \\ &= - \sum_i p_i\log p_i {}- \sum_j q_j\log q_j \\ &= S(\rho_A)+S(\sigma_B), \end{aligned}$$

where $\sum_i p_i=\sum_j q_j=1$ was used in the third line.

6. Consider the equally weighted ensemble $\{\lvert0\rangle,\lvert+\rangle\}$. Compute the density operator, its eigenvalues, and its entropy in bits. Compare with the one-bit entropy of the preparation label.

Solution

The density operator is

$$\begin{aligned} \rho &= \frac12 \lvert0\rangle\langle0\rvert + \frac12 \lvert+\rangle\langle+\rvert \\ &= \begin{pmatrix} 3/4&1/4\\ 1/4&1/4 \end{pmatrix}. \end{aligned}$$

Its trace is $1$ and its determinant is $1/8$, so its eigenvalues are

$$\lambda_\pm = \frac12 \left( 1\pm\frac{1}{\sqrt2} \right).$$

Therefore

$$\begin{aligned} S(\rho) &= h_2 \left[ \frac12 \left( 1+\frac{1}{\sqrt2} \right) \right] \\ &\approx 0.601\ \text{bits}. \end{aligned}$$

The preparation label has entropy one bit, but the two prepared states are not orthogonal and cannot be perfectly distinguished. Discarding the label leaves less than one bit of spectral entropy.

7. For the Bell state $\lvert\Phi^+\rangle$, compute the entropy of the joint state and of each one-qubit reduced state.

Solution

The joint density operator is a rank-one projector, so

$$S(\rho_{AB})=0.$$

Tracing either qubit gives

$$\rho_A=\rho_B=\frac I2.$$

Hence

$$S(\rho_A) = S(\rho_B) = 1\ \text{bit}.$$

The whole is pure while the parts are mixed because the qubits are entangled. For a pure bipartite state, the equal reduced entropies are the entanglement entropy.

8. A two-level Gibbs state has energies $0$ and $\Delta>0$. Find its entropy in nats and its limits as $\beta\Delta\to0$ and $\beta\Delta\to\infty$.

Solution

The partition function and excited-state probability are

$$Z=1+e^{-\beta\Delta}, \qquad p_1 = \frac{e^{-\beta\Delta}}{1+e^{-\beta\Delta}}.$$

The ground-state probability is $p_0=1-p_1$, so

$$S(\rho_\beta) = - p_0\ln p_0 - p_1\ln p_1.$$

Equivalently, with natural logarithms,

$$S(\rho_\beta) = \beta\Delta\,p_1 + \ln \bigl( 1+e^{-\beta\Delta} \bigr).$$

As $\beta\Delta\to0$, the probabilities approach $(1/2,1/2)$ and $S\to\ln2$. As $\beta\Delta\to\infty$, the ground-state probability approaches one and $S\to0$.