Test and measurement automation in defence manufacturing

This record does not share the customer’s name, the product description or the technical content of the test itself. What follows is only the engineering approach to the test infrastructure; which quantity was measured against which limit, and which product was tested, are out of scope.

Starting point

End-of-line testing was a job a qualified technician worked through by hand at a bench. The technician adjusted the power supply manually, wrote the value read off a multimeter and an oscilloscope onto a paper form, signed the form and filed it. The order of the test steps was defined in a procedure document; which revision of that document was actually on the bench depended on the technician’s folder.

That had three concrete problems. Repeatability: when two technicians tested the same product, settling times and set points were not identical. Traceability: when a unit misbehaved in the field, answering “on what date, with which instrument, under which procedure revision and by whom was this serial number tested” meant searching a paper archive by hand. Record integrity: because the measured value reached the form through a human hand, there was no way to separate a transcription error from a genuine deviation.

The system built

The architecture was built as four layers.

Test executive. Test steps were not embedded in code; they are held as a versioned test definition made of sequential steps. Each step’s type (set, measure, wait, decide), its parameters and its pass limits are part of the definition. The software interprets and executes that definition. Why this way: when a new product variant arrives, the software is not recompiled — a test definition is added. The software itself stays a validated component.

Instrument layer. Communication with programmable power supplies, digital multimeters, oscilloscopes and load units through a class-based driver interface. Two instruments from different vendors making the same measurement look identical to the layer above. Instrument calibration dates are held in the system; a test cannot be started with an instrument whose calibration has expired.

Record and audit trail. Every test run is written to the database along with the serial number, the test definition version, instrument identities and calibration status, raw measured values, the pass/fail decision and the operator identity. Records are append-only: if a result has to be corrected, the old record is not deleted — a justified correction record is added.

Authorisation and reporting. An operator runs a test; changing a test definition, editing limits and approving an invalid result are separate roles. The test report is generated from the record and contains no hand-typed field.

For the analytics side of measurement data see data acquisition and analytics, and for the electrical and panel side of the test bench see electrical and electronics.

The hard part

Timing caused the most trouble. The test executive runs on a general purpose operating system, while some steps required a narrow time window between stimulus and measurement. The delay expected in software was a few milliseconds; the actual delay drifted with operating system load. The result: the same unit passed on some runs and landed just outside the limit on others — for a reason that had nothing to do with the unit itself.

Isolating the root cause took more than the software log. The instrument’s own timestamp was compared against the software’s timestamp, and the drift became visible there.

The fix was to take the timing-critical steps out of software: triggering and the measurement window moved to the hardware side, onto a deterministic control unit. The software no longer says “measure” — it says “start this measurement sequence on this condition” and collects the result. Steps with a tight timing tolerance were also marked as a distinct type in the test definition; that type cannot be executed without hardware triggering, so the failure class is closed structurally.

Measurement and acceptance

Two metrics are tracked: test duration per unit and first pass yield. Both are measured before and after the system, on the same sample class and with the same counting rule — how retests are counted is part of the method, not left open to later argument.

On the software side measurement repeatability was also verified: the same sample was run repeatedly under the same test definition and the spread of the readings examined. That measures the noise of the test system itself, not product quality.

Acceptance ran as FAT in the workshop (with known-good reference samples and samples deliberately pushed outside the limits — the system rejecting them has to be verified too), and SAT on site inside the real production flow with a signed protocol. A separate check covered the audit trail: starting from a randomly chosen serial number, the chain was walked through to the test record and on to the instrument’s calibration certificate to show it was complete.