Machine Vision Threshold Selection with Precision–Recall Curves
Choose an operating threshold from false-accept and false-reject costs, representative score distributions, precision–recall behaviour, and production monitoring.
A classifier uses the default 0.5 threshold because its output looks like a probability. The chosen operating point rejects too many good parts while still missing rare defects because class prevalence and decision costs were never considered.
This article presents a vendor-neutral engineering method. Worked figures are hypothetical and must be replaced by evidence from the actual line.
What you will learn
Understand the physical and data-processing limits behind machine vision threshold selection.
Translate an inspection need into measurable component and system requirements.
Calculate or test the variable that most strongly controls reliability.
Recognize common integration mistakes before commissioning.
Validate the final method under representative production conditions.
Technical foundation
Scores are not decisions
A model produces a score; a threshold maps it to pass, reject, review, or invalid. The default numerical cutoff has no universal business meaning.
Precision and recall depend on operation
Recall measures detected positives; precision measures how many positive calls are correct and changes with prevalence. ROC shows true-positive versus false-positive rate.
Threshold tuning needs independent data
Choosing and reporting performance on the same samples produces optimistic results. Use separate tuning and final evaluation evidence grouped by production source.
[Suggested visual: precision–recall curve annotated with feasible thresholds and business guardrails]
Purpose: Clarify the physical or architectural mechanism before component selection.
Required elements: Use labelled rays, axes, signals, components, and the failure mode described; keep scale relationships physically plausible.
Suggested caption: “Precision–recall curve annotated with feasible thresholds and business guardrails.”
Accessible alt text: “Technical diagram of precision–recall curve annotated with feasible thresholds and business guardrails.”
Engineering workflow
1. Define the positive class
What to evaluate: whether positive means defect, reject, or good part.
Why it matters: metric interpretation reverses with class naming.
How to measure or calculate it: write the confusion matrix in factory language. Record the input conditions and keep the same method when comparing alternatives.
Trade-off: stakeholders may use opposite conventions. What commonly goes wrong: reporting precision without class definition.
2. Assign decision costs and constraints
What to evaluate: escape severity, scrap, review capacity, downtime, and contractual limits.
Why it matters: the best threshold is an operating decision.
How to measure or calculate it: quantify a cost matrix or hard guardrails. Record the input conditions and keep the same method when comparing alternatives.
Trade-off: costs may be uncertain and asymmetric. What commonly goes wrong: maximizing accuracy by default.
3. Build representative score sets
What to evaluate: good, defect classes, boundary cases, lots, and drift conditions.
Why it matters: curves reflect only the evaluated distribution.
How to measure or calculate it: split tuning and final sets by part, lot, and run. Record the input conditions and keep the same method when comparing alternatives.
Trade-off: rare defects require deliberate sampling. What commonly goes wrong: randomly splitting near-duplicate images.
4. Plot relevant curves
What to evaluate: precision–recall, ROC, false-accept/reject, and threshold tables.
Why it matters: different plots expose different trade-offs.
How to measure or calculate it: calculate metrics at every candidate threshold. Record the input conditions and keep the same method when comparing alternatives.
Trade-off: ROC can look strong on imbalanced data. What commonly goes wrong: choosing from AUC alone.
5. Choose with guardrails
What to evaluate: maximum escape rate, minimum recall by critical class, and review capacity.
Why it matters: one aggregate optimum may violate a safety-critical class.
How to measure or calculate it: filter thresholds by constraints before cost optimization. Record the input conditions and keep the same method when comparing alternatives.
Trade-off: conservative limits increase false rejects. What commonly goes wrong: selecting the mathematical maximum F1 blindly.
6. Lock and monitor
What to evaluate: threshold version, score distributions, prevalence, drift, and retraining triggers.
Why it matters: production distribution changes the operating point.
How to measure or calculate it: trend quantiles and labelled audit samples. Record the input conditions and keep the same method when comparing alternatives.
Trade-off: frequent threshold changes weaken traceability. What commonly goes wrong: auto-adjusting thresholds without approval.
Worked example
Hypothetical threshold table:
| Threshold | Defect recall | Precision | False reject rate |
|---|---|---|---|
| 0.30 | 99.0% | 62.0% | 3.8% |
| 0.50 | 96.0% | 78.0% | 1.7% |
| 0.70 | 88.0% | 91.0% | 0.6% |
If the hard requirement is at least 95% defect recall, 0.70 is infeasible regardless of its precision. Choose between remaining thresholds using class-specific escape limits, scrap cost, and independent final validation.
[Suggested visual: score distributions for good and defective parts with threshold and drift shift]
Purpose: Turn the engineering workflow into a resource that can be used during commissioning and review.
Required elements: Include the inputs, decision points, measurable outputs, acceptance boundary, and major failure branches.
Suggested caption: “Score distributions for good and defective parts with threshold and drift shift.”
Accessible alt text: “Technical diagram of score distributions for good and defective parts with threshold and drift shift.”
Practical decision aid
| Tool | Shows | Limitation |
|---|---|---|
| Confusion matrix at threshold | actual operating counts | one point only |
| Precision–recall curve | positive-call quality versus coverage | prevalence-dependent |
| ROC curve | TPR versus FPR | can look optimistic for rare positives |
| Cost curve/table | business consequence | cost assumptions required |
| Score histogram | separation and drift | does not set policy alone |
Use this table to choose the next experiment, not as a universal component recommendation. Record actual settings, part variants, and evidence beside the decision.
Common mistakes and how to prevent them
Using 0.5 automatically. Operating costs are ignored. Prevent it by tune explicitly.
Maximizing accuracy. Majority class dominates. Prevent it by use class and cost metrics.
Choosing on test data. Final estimate is optimistic. Prevent it by separate tuning and evaluation.
Using AUC as operating proof. No threshold is selected. Prevent it by report point metrics.
Ignoring prevalence. Precision changes in production. Prevent it by evaluate realistic mix.
Changing thresholds without versioning. Traceability breaks. Prevent it by approve and record changes.
Validate under production conditions
Use a held-out, group-separated dataset and then a monitored production trial. Report confusion matrices and confidence intervals at the selected threshold by critical defect class, lot, environment, and time; retain score distributions for drift monitoring.
A defensible validation set includes representative acceptable parts, defective parts, boundary cases, and nuisance variation. Repeat complete part presentations rather than processing one stored image many times. Include environmental extremes, line-speed limits, start-up and warm-up, maintenance states, interface faults, and long-duration operation where relevant.
Predefine acceptance criteria for false accepts, false rejects, invalid acquisitions, repeatability, cycle time, and recovery. Preserve raw counts and denominators. After release, trend leading indicators and audit labelled samples so that drift is detected before it becomes a customer escape.
Key takeaways
A model score needs an engineered operating threshold.
Define class meaning and business constraints first.
Precision–recall is useful for rare defects.
Tune and evaluate on independent groups.
Version thresholds and monitor score drift.
Follow this Hashnode blog for more practical industrial machine-vision engineering, and connect with Kivanc Ekici on LinkedIn. To explore relevant technology information, visit Algomedi.
Frequently asked questions
Is 0.5 the correct threshold for a probability output?
Only if calibration, prevalence, and decision costs make it appropriate; it is not a universal rule.
When is precision–recall preferred to ROC?
It is often more informative when the positive class is rare and false positive burden matters.
Should F1 be maximized?
Only when equal precision/recall weighting matches business constraints.
Can the threshold be tuned on the test set?
No; reserve independent data for final performance estimation.
How is threshold drift monitored?
Trend score distributions and periodically label representative audit samples under change control.