Variational Optimization

Variational optimization minimizes the energy over a parameterized family of states. Unlike a linear Rayleigh–Ritz calculation, the trial manifold can be curved and the objective nonconvex. A vanishing numerical gradient establishes local stationarity within the chosen family; it does not establish the true ground state.

Helpful background. The variational principle supplies the ground-state bound, and Rayleigh–Ritz covers linear coefficients.

Energy gradients on a trial manifold

For a differentiable, possibly unnormalized state $|\psi(\boldsymbol\theta)\rangle$ with real parameters, define

$$E=\frac{\langle\psi|H|\psi\rangle}{N}, \qquad N=\langle\psi|\psi\rangle>0.$$

Differentiating numerator and denominator, for parameter-independent Hermitian $H$, gives

$$\partial_a E =\frac{2\operatorname{Re} \langle\partial_a\psi|(H-E)|\psi\rangle}{N}.$$

The residual $(H-E)|\psi\rangle$ therefore drives optimization only through directions accessible to the ansatz. The gradient may vanish even when the residual is large, because the missing correction lies outside the tangent space. Energy variance, $\langle H^2\rangle-\langle H\rangle^2$, gives a complementary diagnostic when it is affordable.

Normalization and global phase are redundant directions. Removing them improves conditioning. Positive widths can be parameterized as $\alpha=e^u$, avoiding inadmissible negative values. For orthonormal orbitals, unitary orbital rotations preserve the constraint more reliably than unconstrained entries followed by ad hoc repair.

Gradient checks and optimization steps

Before relying on analytic or automatic derivatives, compare each directional derivative with

$$D_h=\frac{E(\boldsymbol\theta+h\boldsymbol v) -E(\boldsymbol\theta-h\boldsymbol v)}{2h}.$$

In a smooth deterministic calculation, truncation error is $O(h^2)$ while roundoff contributes approximately $O(\epsilon_{\rm mach}/h)$. A sweep in $h$ should reveal a useful plateau. Differentiating through an incompletely converged internal eigensolver can give a different derivative from the converged physical objective.

Use a line search or trust region to control updates; rescale parameters with very different units. Curvature information can accelerate convergence, but nearly redundant parameters make the Hessian ill-conditioned. A state-space metric can expose this redundancy:

$$G_{ab}=\operatorname{Re}\!\left[ \langle\partial_a\bar\psi|\partial_b\bar\psi\rangle -\langle\partial_a\bar\psi|\bar\psi\rangle \langle\bar\psi|\partial_b\bar\psi\rangle\right],$$

where $\bar\psi$ is normalized. Solving $G\,\delta\theta=-\eta\nabla E$ requires regularization when $G$ has small eigenvalues. Record that regularization, since it changes the update.

A width parameter with an exact minimum

For the dimensionless harmonic oscillator and Gaussian width $\alpha=e^u$,

$$E(u)=\frac{e^u+e^{-u}}4=\frac12\cosh u, \qquad E'(u)=\frac12\sinh u.$$

The minimum is $u=0$, with $E=1/2$ and curvature $E''=1/2$. This tests gradients and positivity constraints simultaneously. Adding a displacement $x_0$ contributes $x_0^2/2$, so the minimum must also recover the parity-symmetric center.

For an anharmonic potential $x^2/2+gx^4$, the same Gaussian gives $E(\alpha)=\alpha/4+1/(4\alpha)+3g/(4\alpha^2)$ for $g\geq0$. Its stationary equation is $\alpha^3-\alpha-6g=0$. Solving it checks an optimizer against an independent scalar equation while leaving an honest ansatz error.

Convergence and local minima

Require agreement of objective, gradient, constraint residual, and repeat starts. Compare an expanded ansatz and selected observables. A smaller energy alone can reflect a better approximation, a broken normalization, or inconsistent quadrature. Stochastic objectives require statistical convergence criteria and belong with variational Monte Carlo; a deterministic line search cannot interpret sampling noise as exact curvature.

Exercises

For small positive $g$, obtain the optimized Gaussian width and energy through first order in $g$.

Solution

Set $\alpha=1+ag+O(g^2)$ in $\alpha^3-\alpha-6g=0$. Then $2a-6=0$, so $\alpha=1+3g+O(g^2)$. Because the unperturbed energy is stationary at $\alpha=1$, the width change contributes only at second order; $E=1/2+3g/4+O(g^2)$.

References

• Nocedal, J., and S. J. Wright, Numerical Optimization, 2nd ed., Springer, 2006, doi:10.1007/978-0-387-40065-5 — line searches, conditioning, and constrained optimization.
• Sorella, S., “Generalized Lanczos algorithm for variational quantum Monte Carlo,” Physical Review B 64, 024512 (2001), doi:10.1103/PhysRevB.64.024512 — state-space updates and variational optimization.