Variational Parameters

A variational parameter is a coordinate on a chosen family of trial states. Widths, orbital exponents, mixing angles, correlation lengths, tensor entries, and circuit rotations all play this role. Optimizing them means finding the lowest Rayleigh quotient accessible to that family, not solving the unrestricted Hilbert-space problem.

Required background. The Variational Principle supplies the upper bound; Trial Wavefunctions explains ansatz design, and Calculus of Variations supplies stationarity conditions.

Parameterized Rayleigh Quotient

Let

$$\boldsymbol\theta = (\theta^1,\ldots,\theta^p) \in \mathcal P$$

be real parameters and let $\lvert\psi(\boldsymbol\theta)\rangle$ be a differentiable family of nonzero admissible states. The strong-residual formulas below assume $\psi\in D(H)$ and enough regularity to differentiate its energy. More generally a family differentiable in the form norm has the same quotient derivative with $\langle\partial_a\psi|H|\psi\rangle$ replaced by $q_H(\partial_a\psi,\psi)$; a Hilbert-space residual need not exist. For a fixed Hamiltonian $H$, define

$$E(\boldsymbol\theta) = \frac{ \langle\psi(\boldsymbol\theta)\rvert H \lvert\psi(\boldsymbol\theta)\rangle }{ \langle\psi(\boldsymbol\theta) \vert \psi(\boldsymbol\theta)\rangle }.$$

The parameter optimum is

$$E_{\mathrm{opt}} = \inf_{\boldsymbol\theta\in\mathcal P} E(\boldsymbol\theta).$$

The distinction between minimum and infimum matters. A parameter can run toward a boundary or infinity while the energy approaches a limiting value that is never attained inside $\mathcal P$. Compact parameter ranges do not arise automatically.

Three spaces should remain conceptually separate:

1. Parameter space $\mathcal P$, where the optimizer moves.
2. Trial manifold $\mathcal M$, the set of physical rays represented by the ansatz.
3. Hilbert space, which contains both $\mathcal M$ and states the ansatz cannot represent.

Different parameter values can represent the same physical ray. Overall phase, normalization, periodic angles, tensor-network gauge transformations, and redundant basis functions all create such nonuniqueness.

Derivative of the Energy

Write

$$N(\boldsymbol\theta) = \langle\psi\vert\psi\rangle, \qquad \lvert\partial_a\psi\rangle = \frac{\partial}{\partial\theta^a} \lvert\psi\rangle.$$

For fixed $H$, quotient differentiation gives

$$\partial_a E ={} \frac{ \langle\partial_a\psi\rvert H\lvert\psi\rangle + \langle\psi\rvert H\lvert\partial_a\psi\rangle }{N} - E \frac{ \langle\partial_a\psi\vert\psi\rangle + \langle\psi\vert\partial_a\psi\rangle }{N}.$$

Introduce the residual

$$\lvert r\rangle = (H-E)\lvert\psi\rangle.$$

Then the gradient component has the compact form

$$\partial_a E = \frac{2}{N} \operatorname{Re} \langle\partial_a\psi\vert r\rangle.$$

This formula is valid whether or not the ansatz is explicitly normalized. It also reveals the central geometry: at a stationary point, the residual is real-orthogonal to every parameter tangent,

$$\operatorname{Re} \langle\partial_a\psi\vert r\rangle = 0, \qquad a=1,\ldots,p.$$

The residual need not vanish. Optimization only removes its components visible inside the tangent space of the trial family.

Normalized states

If $N=1$ for every parameter value, differentiation of the normalization condition gives

$$2\operatorname{Re} \langle\psi\vert\partial_a\psi\rangle = 0.$$

The tangent can still contain a purely imaginary component parallel to $\lvert\psi\rangle$, corresponding to a parameter-dependent phase. Because

$$\langle\psi\vert r\rangle = 0,$$

that phase direction does not affect the energy gradient.

Complex parameters

For complex parameters, either split each one into real and imaginary parts or use Wirtinger derivatives with $z^a$ and $z^{a*}$ treated as independent coordinates. For a real-valued differentiable objective, the two Wirtinger derivatives are conjugate: setting either entire complex derivative to zero imposes both real stationarity conditions. Treating the real energy as a holomorphic function, or varying only the real part of a complex parameter, is generally incorrect.

Tangent Vectors and the State-Space Metric

Normalized vectors that differ only by overall phase represent the same physical state. Remove the phase-and-normalization component of a parameter derivative by defining

$$\lvert D_a\psi\rangle = \left( I- \lvert\psi\rangle\langle\psi\rvert \right) \lvert\partial_a\psi\rangle.$$

For normalized states,

$$\partial_a E = 2\operatorname{Re} \langle D_a\psi\vert r\rangle.$$

The real part of the pulled-back quantum geometric tensor defines a metric on nonredundant parameter directions:

$$G_{ab} = \operatorname{Re} \langle D_a\psi\vert D_b\psi\rangle.$$

Equivalently,

$$G_{ab} ={} \operatorname{Re} \bigl[ \langle\partial_a\psi \vert\partial_b\psi\rangle - \langle\partial_a\psi\vert\psi\rangle \langle\psi\vert\partial_b\psi\rangle \bigr].$$

This metric measures how much the physical state changes, rather than how far the raw coordinates move. It is positive semidefinite. A zero eigenvalue means no first-order change of the physical ray along that coordinate direction. It can reflect a true redundancy or a singular parameterization: $\psi(\theta)\propto|0\rangle+\theta^2|1\rangle$ has zero metric at $\theta=0$ even though finite changes matter. Nearly zero eigenvalues indicate weak first-order sensitivity.

The full geometric interpretation is developed in Fubini–Study Geometry. For optimization, the practical lesson is immediate: equal Euclidean steps in two parameters need not produce equal changes in the state.

Geometry-aware updates

Let

$$g_a = \partial_a E.$$

An ordinary gradient step is

$$\delta\theta^a = -\eta g_a.$$

A metric-scaled or natural-gradient step solves

$$\sum_b \left( G_{ab}+\lambda\delta_{ab} \right) \delta\theta^b = -\eta g_a,$$

where $\lambda\geq0$ is a damping parameter. At $\lambda=0$, a pseudoinverse removes null directions. Positive damping instead bounds the amplification of small metric eigenvalues; adding $\lambda I$ also introduces a coordinate-dependent Euclidean scale. Neither operation makes the state metric equal to the energy Hessian.

Natural-gradient language does not create a universal optimizer. The metric can be noisy, singular, expensive to form, or only locally informative. Damping, trust regions, and independent convergence checks remain necessary.

Multiple Parameters and Stationarity

For an unconstrained interior optimum $\boldsymbol\theta_\star$,

$$\nabla_{\boldsymbol\theta}E \big|_{\boldsymbol\theta_\star} = 0.$$

This condition is necessary, not sufficient. It can describe a local minimum, maximum, saddle, or flat manifold of equivalent parameterizations.

Coordinate dependence of gradients

Suppose $\boldsymbol\eta$ is another smooth coordinate system with

$$\theta^a = \theta^a(\boldsymbol\eta).$$

The gradient components transform by the chain rule:

$$\frac{\partial E}{\partial\eta^i} = \sum_a \frac{\partial\theta^a}{\partial\eta^i} \frac{\partial E}{\partial\theta^a}.$$

The statement that the gradient vanishes is coordinate independent at a nonsingular change of variables. The numerical size of the gradient is not. A stopping rule such as $\lVert\nabla E\rVert\lt\varepsilon$ is meaningless until the parameter scaling and norm have been specified.

Linear coefficients as variational parameters

For

$$\lvert\psi\rangle = \sum_j c_j\lvert\phi_j\rangle,$$

stationarity with respect to all real and imaginary coefficient directions gives

$$Hc=ESc.$$

Thus the generalized eigenvalue equation is the exact coefficient-stationarity condition inside a linear trial space. Nonlinear parameters in the basis functions require an additional outer optimization. See the Rayleigh–Ritz Method.

Constraints and Reparameterization

Constraints are often best enforced by construction. An optimizer should not be allowed to step outside the domain where the trial state is meaningful.

Positive parameters

If a width $b$ must satisfy $b\gt0$, write

$$b = b_{\mathrm{ref}}e^\eta, \qquad \eta\in\mathbb R.$$

Then

$$\frac{dE}{d\eta} = b\frac{dE}{db}.$$

At a stationary point,

$$\frac{d^2E}{d\eta^2} = b^2\frac{d^2E}{db^2},$$

because the term $b\,dE/db$ vanishes there. Logarithmic coordinates enforce positivity and make multiplicative scale changes additive.

Normalization

Explicit normalization can be built into the ansatz, or the Rayleigh quotient can remove a free overall scale. Treating normalization as an independent unconstrained parameter creates a flat direction. If one instead minimizes $\langle\psi\rvert H\lvert\psi\rangle$ subject to $\langle\psi\vert\psi\rangle=1$, a Lagrange multiplier produces the eigenvalue equation.

Equality constraints

For constraints

$$C_\mu(\boldsymbol\theta) = 0, \qquad \mu=1,\ldots,q,$$

under a regularity condition such as independent constraint gradients, the first-order condition is

$$\nabla E + J_C^{\mathsf T} \boldsymbol\lambda = 0,$$

where

$$(J_C)_{\mu a} = \frac{\partial C_\mu}{\partial\theta^a}.$$

Equivalently, the gradient must vanish along every tangent direction that preserves the constraints to first order.

Symmetry constraints

Particle exchange, parity, angular momentum, gauge constraints, and boundary conditions should usually be built into the state family rather than imposed through a large energy penalty. A finite penalty changes the objective and can leave symmetry contamination. Projection methods can restore a symmetry, but their normalization and parameter derivatives must then be included consistently.

Hessians and Local Curvature

The parameter Hessian is

$$K_{ab} = \frac{\partial^2E}{ \partial\theta^a\partial\theta^b }.$$

At an unconstrained local minimum, $K$ is positive semidefinite. Positive definiteness requires an isolated nondegenerate quadratic minimum after redundant directions are removed; even an isolated minimum such as $E(\theta)=E_*+\theta^4$ has zero Hessian. A negative eigenvalue identifies a descent direction, while a zero eigenvalue can signal symmetry, gauge redundancy, an underdetermined ansatz, or a higher-order flat direction.

Curvature at an exact eigenstate

Suppose a normalized ansatz passes through an exact eigenstate $\lvert n\rangle$ at $\boldsymbol\theta_0$. Since the residual vanishes there, the Hessian simplifies to

$$K_{ab} = 2\operatorname{Re} \langle D_a n\rvert (H-E_n) \lvert D_b n\rangle.$$

For the ground state, this quadratic form is nonnegative. For an excited eigenstate, a tangent confined to lower-energy states has negative curvature unless constraints exclude it. A general tangent with both lower- and higher-energy components has curvature set by their energy-weighted sum; lower-state overlap alone does not determine its sign. An excited eigenstate is therefore typically a saddle of the unrestricted energy functional. Rigorous excited-state bounds use the ordered subspace construction developed in Upper Bounds and the Min–Max Principle.

Stiff and soft directions

Let the nonzero Hessian eigenvalues near a minimum be

$$0\lt\kappa_{\min} \leq \kappa_i \leq \kappa_{\max}.$$

The ratio

$$\operatorname{cond}(K) = \frac{\kappa_{\max}}{\kappa_{\min}}$$

measures local anisotropy in the chosen coordinates. A large ratio creates a narrow valley: steps stable in the stiff direction make painfully slow progress along the soft direction. This is an optimization-conditioning problem, not automatically evidence of a poor ansatz.

Elongated contours indicate very different curvatures along two parameter combinations. Unscaled coordinate-gradient steps can zigzag across a stiff direction; curvature or state-metric information can propose a better-scaled local step. Neither path changes the trial family’s ansatz error.

The numerical interpretation of condition numbers and stability is developed in Conditioning and Stability.

Scale Analysis Before Optimization

Physical dimensions should be removed before handing a problem to a numerical optimizer. If one parameter is measured in inverse ångströms and another in joules, an ordinary Euclidean gradient combines incomparable coordinate components.

A useful workflow is:

1. identify natural length, energy, and time scales from the Hamiltonian;
2. express the ansatz in dimensionless variables;
3. center parameters near physically plausible values;
4. use logarithmic variables for positive scales spanning orders of magnitude;
5. inspect Hessian or metric eigenvalues after the transformation.

Good scaling does not alter the physical trial family. It changes the chart used to search it.

Scale Translation: Quartic Gaussian Width

The quartic oscillator provides a compact example of why scaling should precede optimization. For $H=P^2/(2m)+m\omega^2X^2/2+\lambda X^4$ with $\lambda\ge0$ and a Gaussian proportional to $e^{-x_{\rm phys}^2/(2b^2)}$, define

$$x = \frac{b}{\sqrt{\hbar/(m\omega)}}, \qquad \gamma = \frac{\lambda\hbar}{m^2\omega^3},$$

the Gaussian Rayleigh quotient becomes

$$\frac{E(x)}{\hbar\omega} = \frac{1}{4x^2} + \frac{x^2}{4} + \frac{3\gamma x^4}{4}.$$

The positive parameter $x$ is dimensionless and order unity at weak coupling. Frequency notation uses $y=\Omega/\omega=x^{-2}$, so the same stationary condition can be written in either chart:

$$-1+x^4+6\gamma x^6=0 \quad\Longleftrightarrow\quad y^3-y-6\gamma=0.$$

The complete normalization, minimization, weak- and strong-coupling limits, upper-bound interpretation, residual analysis, and numerical comparison are developed in Anharmonic Oscillator by Variational Methods. The coordinate lesson needed here is that the physical optimum is unchanged by the smooth reparameterization $y=x^{-2}$, while gradients and curvatures acquire different numerical scales.

Parameter-Dependent Bases

Suppose a nonorthogonal basis depends on a nonlinear parameter $\alpha$ and the optimized coefficients solve

$$H(\alpha)c = E(\alpha)S(\alpha)c,$$

with normalization

$$c^\dagger S c = 1.$$

Along a differentiable normalized eigenbranch of this Hermitian generalized problem,

$$\frac{dE}{d\alpha} = c^\dagger \left( \frac{dH}{d\alpha} - E\frac{dS}{d\alpha} \right) c.$$

At a degeneracy one must first select branches by diagonalizing $H' - ES'$ within the degenerate eigenspace, using its $S$ metric. The $dS/d\alpha$ term is essential. It records the changing basis geometry and is a finite-dimensional relative of a Pulay term. Ignoring it treats a moving nonorthogonal basis as fixed and generally gives the wrong gradient.

If the coefficient eigenproblem is not converged, additional response terms survive. The clean derivative formula relies on stationarity with respect to $c$.

Optimized Energies and the Envelope Principle

Let the Hamiltonian depend on a physical control parameter $\lambda$, and define

$$E_\star(\lambda) = E( \boldsymbol\theta_\star(\lambda), \lambda ).$$

The total derivative is

$$\frac{dE_\star}{d\lambda} = \left. \frac{\partial E}{\partial\lambda} \right|_{\boldsymbol\theta} + \sum_a \frac{\partial E}{\partial\theta^a} \frac{d\theta^a_\star}{d\lambda}.$$

At an unconstrained stationary optimum, the second term vanishes:

$$\frac{dE_\star}{d\lambda} = \left. \frac{\partial E}{\partial\lambda} \right|_{\boldsymbol\theta_\star}.$$

This is why optimized variational energies often have simple first derivatives even though the optimal parameters move. Incomplete optimization, changing constraints, and parameter-dependent bases can restore response terms. The operator-derivative statement and its caveats are treated in the Hellmann–Feynman Theorem.

Choosing a Numerical Optimizer

No optimizer repairs a trial family that cannot represent the needed physics. Within a reasonable family, the method should match the parameter count, smoothness, conditioning, constraints, and noise level.

Method	Useful when	Main caution
Analytic stationarity	One or a few parameters and tractable integrals	Roots can be maxima or saddles
Gradient descent with line search	Gradients are reliable and scaling is moderate	Slow in narrow valleys
Conjugate-gradient or quasi-Newton	Smooth medium-to-large problems	Curvature models can be corrupted by noise
Newton or trust-region method	Accurate Hessian information is affordable	Hessians can be indefinite or singular
Derivative-free search	Very few parameters or nonsmooth evaluations	Scales poorly with dimension
Natural gradient or stochastic reconfiguration	State geometry is important or Monte Carlo samples provide the metric	Metric estimation, damping, and rank control are delicate

Line searches and trust regions

A descent direction is not enough. A line search chooses a step length that actually lowers the objective, while a trust region limits the update to a neighborhood where the local model is credible. In noisy calculations, requiring a decrease much smaller than the estimator uncertainty has no statistical meaning.

Stochastic objectives

In variational Monte Carlo, the energy, gradient, and metric are sample estimates. Reusing correlated samples can reduce the noise in energy differences, but it also complicates uncertainty estimates. Optimization bias and final statistical uncertainty should be assessed with fresh samples. See Variational Monte Carlo Preview.

Automatic differentiation

Automatic differentiation can evaluate exact derivatives of the implemented computational graph. It does not certify that the graph represents the intended Hamiltonian, normalization, boundary conditions, or sampling distribution. Analytic limits and finite-difference spot checks remain valuable.

Exercises

Logarithmic width coordinate

Let $b=b_{\mathrm{ref}}e^\eta$. Derive the first and second derivatives of $E$ with respect to $\eta$, and simplify the second derivative at a stationary point in $b$.

Solution

Because $db/d\eta=b$,

$$\frac{dE}{d\eta} = b\frac{dE}{db}.$$

Differentiating again,

$$\frac{d^2E}{d\eta^2} = b\frac{dE}{db} + b^2\frac{d^2E}{db^2}.$$

At $dE/db=0$,

$$\left. \frac{d^2E}{d\eta^2} \right|_\star = b_\star^2 \left. \frac{d^2E}{db^2} \right|_\star.$$

Thus the sign of the curvature at an interior stationary point is unchanged, while its numerical scale changes.

Why an excited eigenstate is a saddle

Consider a normalized path through $\lvert n\rangle$ whose projected tangent is $\lvert D\psi\rangle=a\lvert m\rangle$, with $a\ne0$ and $E_m\lt E_n$. Use the exact-eigenstate Hessian to determine the curvature along this purely lower-state direction. If a higher eigenstate is also available, explain why $\lvert n\rangle$ is a saddle in the unrestricted state space.

Solution

Take a projected tangent $\lvert D\psi\rangle=a\lvert m\rangle$. The second variation is proportional to

$$2\langle D\psi\rvert (H-E_n) \lvert D\psi\rangle = 2\lvert a\rvert^2 (E_m-E_n) \lt0.$$

Since $E_m-E_n\lt0$, the curvature is negative. The energy decreases by mixing in the lower state, so the excited eigenstate is not a local minimum in the unrestricted state space. A purely higher-state tangent gives positive curvature, establishing the saddle when both directions exist. A symmetry or orthogonality constraint can remove either direction. A tangent mixing lower and higher states instead has a weighted sum of both signs, so its lower-state component alone does not determine the curvature.

Derivative in a moving nonorthogonal basis

Differentiate $Hc=ESc$ with respect to $\alpha$, left-multiply by $c^\dagger$, and use $c^\dagger H=E c^\dagger S$ and $c^\dagger Sc=1$ to derive the parameter-dependent-basis formula.

Solution

Differentiation gives

$$H'c+Hc' = E'Sc+ES'c+ESc'.$$

Left-multiplying by $c^\dagger$ yields

$$c^\dagger H'c + c^\dagger Hc' ={} E'c^\dagger Sc + E c^\dagger S'c + E c^\dagger Sc'.$$

Using $c^\dagger H=E c^\dagger S$ cancels the terms containing $c'$, and $c^\dagger Sc=1$ gives

$$E' = c^\dagger(H'-ES')c.$$

The overlap derivative cannot be discarded when the basis moves.

References

• S.-i. Amari, “Natural Gradient Works Efficiently in Learning”, Neural Computation 10, 251–276 (1998). Foundational account of metric-aware parameter optimization.
• J. Nocedal and S. J. Wright, Numerical Optimization, 2nd ed. (Springer, 2006). Standard reference for line searches, trust regions, quasi-Newton methods, constraints, and conditioning.
• M. Reed and B. Simon, Methods of Modern Mathematical Physics, Vol. IV: Analysis of Operators (Academic Press, 1978). Mathematical background for variational eigenvalue principles and quadratic forms.
• S. Sorella, “Green Function Monte Carlo with Stochastic Reconfiguration”, Physical Review Letters 80, 4558–4561 (1998). Introduces stochastic reconfiguration in a many-body Monte Carlo setting.
• J. Stokes, J. Izaac, N. Killoran, and G. Carleo, “Quantum Natural Gradient”, Quantum 4, 269 (2020). Connects natural-gradient updates to the geometry of parameterized quantum states.
• J. Toulouse and C. J. Umrigar, “Optimization of quantum Monte Carlo wave functions by energy minimization”, Journal of Chemical Physics 126, 084102 (2007). Detailed analysis of stable energy-minimization methods for nonlinear wavefunctions.
• C. J. Umrigar, K. G. Wilson, and J. W. Wilkins, “Optimized Trial Wave Functions for Quantum Monte Carlo Calculations”, Physical Review Letters 60, 1719–1722 (1988). Classic treatment of trial-state optimization in variational Monte Carlo.