Monte Carlo Estimators and Error Checks

Monte Carlo integration estimates an expectation by averaging random observations. Its characteristic $N^{-1/2}$ uncertainty can make high-dimensional integrals tractable, but only when the estimator has a manageable variance and the samples explore the relevant distribution. A small displayed standard error is meaningful only after those assumptions have been examined.

Helpful background. Error budgets separates statistical uncertainty from discretization and model errors. Quantum-specific sampling representations are introduced in many-body quantum Monte Carlo.

An estimator and its variance

For a normalized density $p(x)$, define $\mu=\int p(x)f(x)\,dx$. Independent draws $X_i\sim p$ give

$$\widehat\mu_N=\frac1N\sum_{i=1}^Nf(X_i), \qquad \operatorname{Var}(\widehat\mu_N)=\frac{\sigma_f^2}{N}.$$

The sample variance $s_f^2=(N-1)^{-1}\sum_i(f_i-\widehat\mu_N)^2$ estimates $\sigma_f^2$ when it is finite. Report the estimated standard error $s_f/\sqrt N$ separately from the spread of the observations $s_f$. An approximately Gaussian interval needs a suitable central-limit regime; rare heavy tails can make the apparent regime misleading.

For a complex estimator, accumulate real and imaginary means and their covariance. A single unsigned “complex error bar” can hide cancellations or an imaginary bias in a theoretically real observable.

A ground-state Gaussian benchmark

In dimensionless oscillator units, the ground-state density is $p(x)=e^{-x^2}/\sqrt\pi$. It has $\langle x^2\rangle=1/2$ and $\langle x^4\rangle=3/4$. For $f=x^2$,

$$\sigma_f^2=\frac34-\frac14=\frac12, \qquad \operatorname{SE}(\widehat{\langle x^2\rangle}) =\frac1{\sqrt{2N}}.$$

At $N=10^4$ independent draws the expected standard error is about $0.00707$. Repeating independent runs should produce a distribution of estimates consistent with that scale. A single run that lands very close to $1/2$ does not demonstrate superior accuracy.

This benchmark tests the Gaussian generator, observable evaluation, accumulation, and distinction between variance and standard error.

Importance sampling changes the variance

If samples instead come from a normalized density $q(x)>0$ wherever $p(x)f(x)$ contributes, use $g(x)=p(x)f(x)/q(x)$. Then $\mathbb E_qg=\mu$ and

$$\operatorname{Var}_q(g) =\int\frac{p(x)^2|f(x)|^2}{q(x)}\,dx-|\mu|^2$$

for the complex mean-square convention. A proposal with insufficient tail coverage can make this variance infinite, even when the target expectation exists. Concentrating samples near a visually prominent peak is not enough.

If the normalization of $p$ is unknown, the self-normalized weighted ratio is usually used instead. It is consistent under appropriate assumptions but generally biased at finite sample size; reweighting treats its covariance and overlap diagnostics.

A reproducible sampling record

Specify the target, proposal, estimator, random-number generator and seed policy, sample count, and independence assumptions. Separate pilot adaptation from the production sample or use an algorithm whose adaptation is justified. Parallel workers need independent streams or a documented stream-splitting scheme; repeating the same seed on every worker duplicates the sample rather than increasing precision.

Use numerically stable running means and variances for large offsets. Compare independent replicate estimates and inspect tails. If observations come from a Markov chain, replace the independent formula by autocorrelation-aware uncertainty. Thinning does not automatically turn a chain into independent samples.

Exercises

How much must the independent sample count increase to reduce standard error by a factor of three?

Solution

It must increase by a factor of nine because standard error scales as $N^{-1/2}$. Reducing estimator variance by a factor of nine would achieve the same reduction at unchanged sample count. Neither change corrects a biased discretization or a proposal that misses part of the support.

References

• Foulkes, W. M. C., Lubos Mitas, R. J. Needs, and G. Rajagopal, “Quantum Monte Carlo simulations of solids,” Reviews of Modern Physics 73, 33–83 (2001), doi:10.1103/RevModPhys.73.33 — variational and diffusion Monte Carlo with error analysis.
• Hastings, W. K., “Monte Carlo sampling methods using Markov chains and their applications,” Biometrika 57, 97–109 (1970), doi:10.1093/biomet/57.1.97 — stationary sampling and proposal correction.