Evidence and Claim Standards
Evidence is adequate when it supports the particular inference being made. There is no universal ordering in which a theorem always outranks an experiment or a larger device always outranks a smaller one. The useful question is whether the evidence can exclude the relevant alternative explanation under the assumptions of the claim.
Helpful background. Convergence studies separates numerical errors. Quantum certification and benchmarking develops the operational context for validating quantum devices.
State the claim before choosing its evidence
Section titled “State the claim before choosing its evidence”| Claim type | Minimum scientific contract | A common overreach |
|---|---|---|
| Mathematical theorem | Exact statement, quantifiers, hypotheses, and a proof or independently checkable argument | Applying a finite-dimensional theorem to an unbounded operator without domain control |
| Experimental effect | Preparation, controls, measured ensemble, calibration, uncertainty, and rival mechanisms | Treating one signature as a unique identification of a phase |
| Numerical result | Model, discretization, solver, convergence, finite-size/time limits, and independent checks | Treating a small residual as a physical error bar |
| Quantum advantage | Task, accuracy, input/output access, success probability, full resources, and strongest relevant classical comparator | Comparing a complete quantum experiment with a deliberately weak classical method |
| Engineering milestone | Delivered operation, duty cycle, yield, latency, reliability, and integration conditions | Multiplying best-case subsystem results measured under incompatible conditions |
| Foundational inference | Operational predictions and the premises connecting them to an interpretation | Presenting a no-go theorem as excluding alternatives outside its hypotheses |
Eisert and colleagues’ review supports the distinction between different certification tasks and trust models. The examples here are audit questions, not allegations about a particular experiment.
Check the denominator and the selected ensemble
Section titled “Check the denominator and the selected ensemble”Consider an illustrative protocol that succeeds on a fraction of attempts and achieves fidelity on accepted outputs. A high accepted fidelity does not determine the unconditional delivered rate. If attempts occur at rate , the accepted rate is before additional dead time or downstream losses.
For example, and give ten accepted events per second. Raising fidelity by discarding another half of the events changes that rate to five. Whether this is useful depends on the task. Report both quality and acceptance, and specify whether discarded attempts can be recognized before an output is consumed.
Selection is not intrinsically invalid. Heralded quantum protocols rely on it. The error is using a conditional result to support a claim about all attempts, or ignoring a selection rule that is correlated with the property being estimated.
Make uncertainties capable of changing the conclusion
Section titled “Make uncertainties capable of changing the conclusion”Separate statistical uncertainty, calibration uncertainty, model discrepancy, and numerical approximation. An error bar should include the terms relevant to the comparison. If two methods differ by less than their unresolved systematics, ranking them by the central values is unjustified.
For a computational effect , ask whether independent refinement of the basis, time step, physical system size, sampling, and fitting window can change its sign. For an experiment, vary a control that would affect the leading nuisance mechanism differently from the proposed signal. For a theorem, seek the smallest counterexample after removing a hypothesis. Each is a way of testing the inference rather than decorating it with extra precision.
Choose a test that distinguishes explanations
Section titled “Choose a test that distinguishes explanations”Suppose two models fit the training observations equally well. Repeating those observations more often can reduce noise without resolving the ambiguity. Select a new input, intervention, time window, or observable on which their predictions differ. Specify the test before inspecting its outcome when practical; otherwise label exploratory choices and validate them on independent evidence.
A useful graduate project often begins here: identify an ambiguity in a published comparison, reproduce the baseline, and design one discriminating test. The project remains a proposal until its novelty and feasibility are checked. Choosing a Research Direction turns that logic into a bounded work plan.
Report what a result leaves undecided
Section titled “Report what a result leaves undecided”A reproducible calculation establishes that a computational procedure can be recovered; it does not establish that its model is adequate. A failed independent replication can reveal a changed regime, an omitted systematic, a statistical fluctuation, or an error. Investigate the reason before inferring a universal conclusion. The National Academies report gives the terminology for this distinction.
An evidence record should end with a conditional next step: which observation, proof, or comparison would revise the assessment, and in which direction? If no conceivable result could change the conclusion, the record is not yet a useful research assessment.