Image Evidence Storage: Compression, Retention, and Integrity
Size and govern inspection evidence so images remain interpretable, retrievable, and demonstrably unaltered throughout their required life.
A quality investigation requests the image for part 7A-1842, but the database contains a JPEG with no recipe version and a timestamp that does not match the PLC record. Saving images is not the same as preserving evidence. Format, identity, metadata, retention, access, integrity, and recovery must be designed together.
This is a vendor-neutral engineering method. The worked example is hypothetical and must be replaced by measurements from the real product, line, and risk assessment.
What you will learn
Identify the physical, optical, data, or process limit behind machine-vision image evidence storage.
Convert the inspection need into measurable acceptance criteria.
Compare practical architectures and their trade-offs.
Commission the method using repeatable evidence.
Validate the final system under representative production variation.
Technical foundation
Evidence is a bound record, not an isolated file
The image should be associated with part, acquisition, camera, timestamp, recipe, model, calibration, result, measurement units, and disposition. The binding and its authority matter as much as pixel bytes.
Compression choice follows evidential purpose
Lossless formats preserve reconstructed pixel values bit-for-bit. Lossy formats may be acceptable for visual review only after task-specific validation, but can alter edges, texture, code cells, and model inputs. Preserve raw or calibrated pixel meaning where reprocessing is expected.
Integrity, authenticity, and availability differ
A hash can detect byte changes after it was generated; it does not by itself prove who created the record or that the original association was truthful. Access control, trusted timestamps, signatures where justified, replication, and restore testing address different risks.
Related guides on this publication: Machine Vision Traceability: Images, Results, Recipes, and MES and Cybersecurity for Networked Machine Vision Systems and GigE vs USB3 vs CoaXPress: Camera Interface Guide.
Engineering workflow
1. Define evidence and obligations
Evaluate: quality investigation, customer record, model retraining, regulatory need, privacy, contract, legal hold, owner, and retention authority.
Why it matters: purpose controls format, metadata, access, and lifetime.
Measure or calculate: map each record class to users, decisions, retention, and deletion approval. Preserve settings, sample identity, operating state, and the calculation method so alternatives remain comparable.
Trade-off: longer retention supports analysis but raises cost and exposure. Common failure: saving everything forever without an approved purpose.
2. Specify the record schema
Evaluate: part ID, acquisition ID, source, pixel format, dimensions, timestamp basis, recipe/model/calibration hashes, result, disposition, and image URI.
Why it matters: retrieval and interpretation fail when context is missing.
Measure or calculate: reconcile one physical part across PLC, archive, MES, and reject logs. Preserve settings, sample identity, operating state, and the calculation method so alternatives remain comparable.
Trade-off: richer metadata improves investigations but needs version governance. Common failure: using filename order as the only identity.
3. Choose representation and compression
Evaluate: raw sensor bytes, processed image, overlays, PNG, TIFF, validated lossy encoding, bit depth, colour space, and endianness.
Why it matters: format conversions can destroy analysis evidence.
Measure or calculate: round-trip test pixel equality for lossless paths and task performance for any lossy path. Preserve settings, sample identity, operating state, and the calculation method so alternatives remain comparable.
Trade-off: lossless files are larger; lossy files may be unsuitable for reprocessing. Common failure: saving only a screenshot with graphics burned into pixels.
4. Calculate ingestion and capacity
Evaluate: image size distribution, rate, trigger bursts, compression ratio, metadata, indexes, replication, headroom, and retention.
Why it matters: average daily estimates miss peak-write and rebuild loads.
Measure or calculate: load-test sustained and burst rates and project usable capacity including copies. Preserve settings, sample identity, operating state, and the calculation method so alternatives remain comparable.
Trade-off: more headroom costs money but protects line continuity. Common failure: calculating raw capacity without replication or filesystem overhead.
5. Protect integrity and access
Evaluate: hash algorithm, manifest, signing if required, write permissions, immutable tier, audit log, encryption, key ownership, and clock.
Why it matters: records can be altered, substituted, or accessed inappropriately.
Measure or calculate: verify hashes, rejected unauthorised changes, audit events, and key recovery. Preserve settings, sample identity, operating state, and the calculation method so alternatives remain comparable.
Trade-off: strong controls add operational ownership and recovery procedures. Common failure: storing hashes in the same mutable table with no protected authority.
6. Operate retention and recovery
Evaluate: tiering, deletion queue, legal hold, archive migration, backups, restore time, corruption scan, and orphan reconciliation.
Why it matters: an archive that cannot restore or delete correctly is not controlled.
Measure or calculate: perform scheduled sample restores and reconcile database references to objects. Preserve settings, sample identity, operating state, and the calculation method so alternatives remain comparable.
Trade-off: redundancy improves availability but is not a backup of accidental deletion. Common failure: testing writes but never restores or expiry.
Worked example
Hypothetical inspection archive: Each lossless image averages 3.2 MB after compression. The line produces 4 images/s for 20 h/day, 300 days/year, and keeps 90 days online.
Daily image payload = 3.2 MB × 4 × 3,600 × 20 = 921,600 MB ≈ 0.922 TB
90-day payload = 0.922 × 90 = 82.98 TB
With two stored copies and 20% headroom = 82.98 × 2 × 1.20 ≈ 199 TB
Indexes, metadata, filesystem behaviour, burst writes, and compression variability require load tests. A selective policy may store all failures plus sampled passes if that satisfies the approved evidence purpose.
Practical decision aid
| Record strategy | Benefit | Risk to address |
|---|---|---|
| All raw images | maximum reprocessing evidence | capacity and sensitive content |
| All lossless processed images | stable algorithm input record | raw sensor information may be lost |
| Failures plus sampled passes | lower cost with monitoring context | selection bias and missing rare escapes |
| Lossy review copies | small and convenient | not equivalent to analysis pixels |
| Overlay plus source image | human and technical evidence | must bind both versions |
| Hash manifest plus protected storage | change detection and control | key, manifest, and lifecycle ownership |
Use the table to choose the next controlled experiment, not as a universal product recommendation. A component or algorithm is acceptable only when the complete inspection cell meets pre-agreed technical and operational criteria.
Common mistakes and how to prevent them
Saving images without configuration identity. results cannot be reproduced. Prevent it by binding recipe, model, and calibration versions.
Using JPEG by default. artefacts alter task evidence. Prevent it by validating format by purpose or using lossless.
Sizing from average rate only. bursts fill queues and disks. Prevent it by testing peak ingestion and headroom.
Hashing without protected provenance. modified file and hash can be replaced together. Prevent it by protecting manifests and access.
Calling RAID a backup. deletion or corruption propagates. Prevent it by maintaining tested independent recovery.
Retention with no deletion proof. cost and privacy exposure grow. Prevent it by auditing expiry, holds, and disposal.
Validate under production conditions
Verify record creation at maximum rate, burst load, network interruption, storage full, partial write, duplicate ID, clock error, restart, and failover. Retrieve randomly selected records by part ID and confirm pixel decoding, metadata, hashes, recipe and result linkage, and physical disposition. Restore from backup into an isolated environment and measure recovery time. Test authorised expiry, legal hold, format migration, and orphan detection.
Use representative acceptable parts, confirmed defects, boundary samples, and nuisance variation. Repeat complete part presentations rather than processing one stored image many times. Include start-up, warm-up, maximum speed, changeover, maintenance, environmental limits, communication faults, and long-duration operation where relevant.
Define acceptance criteria before reviewing final results. Preserve raw counts and denominators for false accepts, false rejects, invalid acquisitions, timing overruns, and manually reviewed cases. After release, trend leading indicators and conduct labelled audits so deterioration is detected before a customer escape.
Key takeaways
Define why each image is evidence before choosing storage.
Bind pixels to stable identity, configuration, result, and disposition.
Use lossless representation when exact reprocessing matters.
Treat integrity, authenticity, access, backup, and retention as separate controls.
Load-test ingestion and regularly prove retrieval and restore.
Follow this Hashnode blog for more practical industrial machine-vision engineering, and connect with Kivanc Ekici on LinkedIn. To explore relevant technology information, visit Algomedi.
Frequently asked questions
Should inspection images be stored losslessly?
Use lossless storage when exact pixel recovery or reprocessing matters. Validate any lossy format against the specific evidential purpose.
Is PNG always suitable for machine vision?
PNG is lossless and supports several sample types, but confirm bit depth, metadata, encode time, decoder support, and preservation of the camera pixel format.
Does a SHA-256 hash prove an image is authentic?
It detects changes relative to a trusted digest. Authenticity additionally needs protected provenance, access control, and possibly a digital signature.
How do I calculate retention capacity?
Use measured compressed-size distributions, sustained and burst rates, operating hours, copies, indexes, headroom, and rebuild or migration load.
Can I retain only rejected-part images?
Possibly, if the approved purpose permits it. Pass samples are often valuable for drift analysis, audits, and detecting selection bias.