Training effectiveness: when a test result is enough — and when it is not

Define the expected effect before publishing the training

A common problem starts before the test. An organisation launches training without recording how it will recognise success. Completion then becomes the only easy metric.

For an informational change, the objective may be acknowledgement of a revised rule. For a new reporting path, it may be correct recognition of a scenario and escalation route. For a production activity, it may be performing the task against an instruction while observed by an authorised person. These objectives need different evidence.

Match the method to the requirement

A test can check declarative knowledge: sequence, criteria, stop conditions, exceptions and escalation rules. It should not pretend to be an observation of practical work. When an error can cause a material defect, safety incident or serious process interruption, a practical assessment may be needed in addition to a knowledge test.

A useful evidence ladder is:

• acknowledgement — to show that the person received the change;
• knowledge test — to check understanding of defined rules;
• scenario exercise — to check a decision under exceptional conditions;
• task observation — to assess practical performance;
• formal licence or authorisation — where law, a customer or an internal rule requires it.

Not every course needs the strongest control. Excessive formalisation increases cost and can turn the system into a click factory. The criterion should be proportionate to the risk.

An unsuccessful result must trigger an action

If a test matters, define the pass threshold and response to failure before publication. A 70% result is meaningless if nobody knows whether it leads to retraining, clarification, a manager review or temporary restriction from the task.

Preserve the first attempt and later repeats. A history of “failed -> corrective action -> passed” is more informative than replacing everything with one final green status. It shows that the control found a gap and the organisation acted on it.

Keep the training record separate from the competence decision

A training system can document the source, version, assignment, deadline, attempts and result. If the organisation requires observed performance or a supervisor’s decision, that should remain a separate criterion with its own accountable person.

CrewShift should not label someone competent only because they completed material. It can provide part of the trace needed for the competence decision: what was taught, when, against which version and with what result.