How the railway industry uses assessments to promote safety

Class_390_Pendolinos_@_EustonPosted by John Kleeman

I’ve been learning how the railway industry uses assessments to maintain competence. As a frequent train passenger, it’s reassuring that the industry and their regulators carefully enforce a safety first mantra. And how railway and rapid transit companies (sometimes with Questionmark software!) use assessments to check competency for rail workers, especially in safety-critical roles.

There are many government and industry bodies that oversee and promote safety including the US Federal Railroad Administration, the UK Railway Safety & Standards Board (RSSB) and the European Railway Agency.

The RSSB have produced a very useful document: Good practice on Competence Review and Assessments. Here is a table summarized from this document that gives pros and cons of different kinds of assessment.

Type of assessment What it means Pros Cons
Observational Observer watches  participant doing normal work Valid and reliable as it provides first-hand information about performance in real conditionsCaptures information about process and behavior not just outcomes Risk of a ‘special performance’ as someone behaves differently whilst being observedWill not cover emergencies and other non-routine work

Needs good planning

Simulation Participant completes activity which is not real work but replicates real work  closely Provides performance evidence for non-routine workMeasures response to emergencies Heavy on resourcesNeeds careful planning to be valid and reliable
Tests Formal assessment of knowledge on paper or on screen Consistent and objectiveGood for assessing technical knowledge

Cost effective for large numbers of people

Requires skill to make valid and reliable
Work products Examining outcomes of work done, e.g. document written or machine serviced Provides evidence of performance in real work conditions Need to verify authenticityShows outcome but not route to get there
Written reports Report from participant or colleague describing competence on the job Provides evidence to support other methods Need to check authenticityMemory fallible

Requires writing skills

Oral interview Conversation where performance is described and questioned Allows in depth exploration of knowledge and understanding Relies on skills of assessorsHard to make consistent and objective

It’s interesting to see the strengths and weaknesses of different kinds of assessments from such a safety-focused industry and to consider how using different assessments together can cover more ground and reduce safety risks.

Leave a Reply