Not every prototype change deserves another trial. A change is worth retesting when a defined uncertainty remains, the change can be isolated sufficiently to evaluate, and the result could alter the next design, inspection, or sourcing decision. Without those conditions, another trial may add effort without producing more useful evidence.
For prototypes involving supplier samples, ASQ includes prototyping, lab testing, and validation testing among the methods for evaluating supplier samples. NIST separately identifies feedback from the inspection stage back to design as part of its digital-thread work. These are useful factual anchors, but neither statement provides a universal threshold for deciding when another trial is necessary.
Start With the Unresolved Question
A modification is not automatically a reason to run another trial. The case for retesting becomes clearer when the maker can identify:
- The specific uncertainty left by the current evidence
- The expected relationship between the prototype change and the issue being investigated
- The comparison that will show whether the change helped, failed, or had no discernible effect
- The decision that the trial result will inform
If the proposed trial cannot answer any of these questions, the team should first clarify the test purpose. A new prototype version may still be useful, but usefulness alone does not make another trial informative.
Match the Decision to the Evidence Pattern
Different evidence patterns call for different responses. The following is a practical decision aid rather than an ASQ or NIST rule.
| Evidence pattern | Trial decision |
|---|---|
| A verified mismatch is linked to one defined prototype change | Another trial may be justified to confirm the change under comparable conditions |
| Several changes are introduced together | Another trial is not yet informative because the result cannot be attributed to a specific revision |
| Existing results are inconsistent or test conditions are unclear | A repeat may be justified if the next trial is designed to resolve that inconsistency |
| The change has no defined connection to the relevant evaluation | Another trial is unlikely to add useful evidence unless its purpose is documented |
| The outcome cannot change the next design or sourcing decision | Another trial is not justified merely to obtain another data point |
These distinctions prevent a trial from becoming an indefinite search for a preferred result. They also make clear when a trial is unnecessary.
Define the Trial Before Running It
A useful trial record should state the exact prototype revision, the baseline being compared, the source and condition of the samples, and the variables that will remain unchanged. It should also identify the measurements or observations to be recorded and the decision rule to be applied.
Writing the decision rule before seeing the result reduces the risk of redefining success after the prototype has been built. The record should explain both what result would support further development and what result would require another revision, additional evaluation, or a different approach.
For supplier-sample work, the conclusion should remain tied to the samples and conditions actually evaluated. A favorable prototype result should not be treated as proof beyond those conditions.
Feed Inspection Findings Back Into Design
NIST’s digital-thread work identifies the feedback cycle from inspection back to design. A practical application of that idea is to connect each material finding to an explicit design decision:
Inspection finding → design decision → prototype revision → next evaluation
The finding does not automatically require a redesign. It may instead be accepted, deferred, or rejected for a documented reason. In either case, recording the decision prevents the same unresolved question from returning without additional context.
ASQ’s inclusion of prototyping, lab testing, and validation testing reflects the different roles evaluation can play. A prototype can expose a design issue early, while later testing can examine supplier samples under defined conditions. The appropriate choice still depends on the question the maker needs to answer.
What the Maker Must Confirm Separately
The cited ASQ and NIST statements do not specify project acceptance thresholds, test conditions, sample quantities, fees, schedules, contractual rights, or regulatory classifications. They also do not establish that one trial will be sufficient.
The maker must separately confirm those matters through the applicable project specifications, supplier records, contracts, and legal or regulatory materials. If the relevant requirements are unavailable or unclear, the acceptance decision remains unconfirmed rather than being inferred from general guidance.
Another trial is therefore justified by unresolved evidence and decision value—not by the mere desire to test a new version. The essential test is whether a defined change, a usable comparison, and a clear consequence can be specified before the trial begins.