Image-Quality Metrics for Machine Vision Commissioning
Commission focus, contrast, noise, uniformity, and stability with task-linked metrics instead of subjective “looks sharp” approval.
A commissioning image looks crisp on a large monitor, yet the smallest laser mark disappears when the enclosure heats up. Human judgement is useful for finding gross problems but cannot establish repeatable acceptance across cameras, technicians, and time. Task-linked image-quality metrics provide a baseline and maintenance evidence.
This is a vendor-neutral engineering method. The worked example is hypothetical and must be replaced by measurements from the real product, line, and risk assessment.
What you will learn
Identify the physical, optical, data, or process limit behind image-quality metrics.
Convert the inspection need into measurable acceptance criteria.
Compare practical architectures and their trade-offs.
Commission the method using repeatable evidence.
Validate the final system under representative production variation.
Technical foundation
Camera characterisation and cell image quality differ
EMVA 1288 defines controlled methods for camera and sensor characteristics. A commissioned cell additionally includes lens, lighting, window, part, geometry, exposure, processing, and environment. Use camera specifications as inputs, not substitutes for cell tests.
Metrics must be task and ROI specific
Sharpness, contrast, noise, saturation, uniformity, blur, and clipping matter differently for a code reader, edge gauge, colour classifier, or texture detector. Measure the features and backgrounds that drive the decision.
A metric is useful only with controlled interpretation
Variance or gradient energy can rise from noise as well as focus. Contrast can increase while clipping destroys detail. Combine metrics, lock acquisition settings, and confirm correlation with inspection outcomes.
Related guides on this publication: Machine Vision MTF: Match Lens Resolution to Pixel Size and Flat-Field Correction for Machine Vision and Machine Vision FAT and SAT: Build an Acceptance Test That Works.
Engineering workflow
1. Define task-critical image evidence
Evaluate: smallest feature, edge, texture, code cell, colour difference, dynamic range, and decision margin.
Why it matters: quality means fitness for the inspection, not aesthetic appeal.
Measure or calculate: map each algorithm input to a measurable image signal. Preserve settings, sample identity, operating state, and the calculation method so alternatives remain comparable.
Trade-off: task-specific metrics are credible but less reusable. Common failure: choosing a generic sharpness number with no acceptance link.
2. Lock and record acquisition state
Evaluate: exposure, gain, gamma, black level, pixel format, aperture, focus, lighting current, trigger, temperature, and processing.
Why it matters: automatic controls can hide drift and destroy comparability.
Measure or calculate: read back settings and store them with baseline images. Preserve settings, sample identity, operating state, and the calculation method so alternatives remain comparable.
Trade-off: locked settings reduce adaptability but expose physical change. Common failure: baselining while auto exposure is enabled.
3. Measure focus and sampling
Evaluate: edge spread, MTF, gradient, frequency target, feature width, field zones, and depth positions.
Why it matters: focus may vary across field, depth, temperature, or lens mount.
Measure or calculate: record task-linked sharpness in a grid and at depth limits. Preserve settings, sample identity, operating state, and the calculation method so alternatives remain comparable.
Trade-off: formal MTF is stronger evidence but needs controlled targets. Common failure: checking focus only at the image centre.
4. Measure signal and noise
Evaluate: foreground/background mean, standard deviation, contrast-to-noise ratio, saturation, dark offset, and temporal noise.
Why it matters: threshold and classifier margins depend on distributions, not means alone.
Measure or calculate: capture repeated frames and calculate ROI distributions. Preserve settings, sample identity, operating state, and the calculation method so alternatives remain comparable.
Trade-off: longer exposures improve photons but increase blur and cycle constraints. Common failure: using an image histogram without separating relevant regions.
5. Measure spatial and temporal stability
Evaluate: uniformity, shading, dead pixels, flicker, frame-to-frame shift, warm-up, vibration, and contamination.
Why it matters: stable initial quality can drift during production.
Measure or calculate: trend metrics through warm-up and environmental extremes. Preserve settings, sample identity, operating state, and the calculation method so alternatives remain comparable.
Trade-off: more health metrics improve diagnosis but require false-alarm control. Common failure: setting limits from one short cold run.
6. Link image limits to inspection outcomes
Evaluate: challenging samples, threshold margins, false decisions, invalid images, and challenge artefacts.
Why it matters: a statistically changed image metric matters only if its operational effect is understood.
Measure or calculate: perturb focus, light, exposure, and contamination until inspection degradation is observed. Preserve settings, sample identity, operating state, and the calculation method so alternatives remain comparable.
Trade-off: guard limits reduce surprise but may stop before actual failure. Common failure: accepting image-quality limits solely because all commissioning samples passed.
Worked example
Hypothetical dark-mark inspection: Mark and background ROIs have means 92 and 138 grey levels. Their temporal standard deviations are 5 and 7. Define a simple pooled contrast-to-noise ratio:
CNR = |138 - 92| / sqrt(5² + 7²)
= 46 / 8.60 = 5.35
After a dirty-window challenge, the means become 105 and 132 with standard deviations 7 and 8, giving CNR = 27 / 10.63 = 2.54. These numbers are only useful after validation links CNR ranges to mark-reading performance.
Practical decision aid
| Metric | What it can indicate | What can mislead it |
|---|---|---|
| Edge/gradient sharpness | focus or motion blur | noise and high-contrast clutter |
| MTF or edge-spread result | spatial-frequency transfer | target, method, sampling, and field position |
| CNR | feature separation relative to noise | ROI choice and nonstationary texture |
| Saturation fraction | lost highlight or shadow detail | irrelevant bright fixture regions |
| Uniformity | lighting or optical shading | intended part reflectance variation |
| Frame-to-frame registration | vibration or timing instability | real part motion |
Use the table to choose the next controlled experiment, not as a universal product recommendation. A component or algorithm is acceptable only when the complete inspection cell meets pre-agreed technical and operational criteria.
Common mistakes and how to prevent them
Approving by monitor appearance. display scaling and human judgement vary. Prevent it by using stored numerical baselines.
One sharpness ROI. field tilt or decentring is missed. Prevent it by mapping the full usable field.
Automatic exposure during tests. brightness stays stable while settings drift. Prevent it by locking and recording acquisition.
Using variance as focus alone. noise raises the metric. Prevent it by combining noise and task-edge measures.
No warm-up test. focus and light settle after acceptance. Prevent it by testing from cold start through steady state.
Limits unrelated to decisions. alarms are arbitrary. Prevent it by correlating controlled degradation with performance.
Validate under production conditions
Build baselines with a controlled target and representative production samples at all critical field and depth locations. Test cold start, warm-up, maximum exposure rate, vibration, ambient light, temperature, window contamination, focus offsets, illumination reduction, and cable or packet faults. Repeat frames and presentations. Relate each metric to algorithm margins and false decisions, then establish warning and invalid-image limits with challenge-part checks.
Use representative acceptable parts, confirmed defects, boundary samples, and nuisance variation. Repeat complete part presentations rather than processing one stored image many times. Include start-up, warm-up, maximum speed, changeover, maintenance, environmental limits, communication faults, and long-duration operation where relevant.
Define acceptance criteria before reviewing final results. Preserve raw counts and denominators for false accepts, false rejects, invalid acquisitions, timing overruns, and manually reviewed cases. After release, trend leading indicators and conduct labelled audits so deterioration is detected before a customer escape.
Key takeaways
Define image quality from the inspection task.
Record acquisition state with every baseline.
Measure focus, signal, noise, clipping, uniformity, and stability by ROI.
Use controlled degradation to connect metrics to inspection risk.
Trend a small diagnostic set that operators can act on.
Follow this Hashnode blog for more practical industrial machine-vision engineering, and connect with Kivanc Ekici on LinkedIn. For related machine-vision and automation information, visit ITAGE.
Frequently asked questions
What is the best focus metric for machine vision?
There is no universal best metric. Use a task-relevant edge or frequency response, map it across the field, and separate focus from noise.
Does EMVA 1288 certify the complete vision cell?
No. It characterises cameras or sensors under defined conditions; the installed cell adds optics, lighting, mechanics, environment, and processing.
What CNR is required?
No universal value. Required separation depends on the algorithm, nuisance distributions, defect severity, and validated threshold margin.
How often should image quality be checked?
At commissioning, after controlled changes or maintenance, and continuously or periodically according to drift risk and challenge-part strategy.
Should baseline images be compressed?
Preserve a lossless or otherwise validated representation with settings and identity when pixel-level comparison is required.