Where do you get evidence for training effectiveness? Ideas from the A-model
Posted by John Kleeman
How do you design a training intervention to improve performance? And how do you gather effective evidence to show that the intervention is effective?
In a couple of previous articles, Problem, Performance and Program in the A-model and A whistle-stop tour round the A-model, I’ve described how the A-model helps you do this by separating Problem, Performance and Program; and then by working through these to analyze and design the program and then assess and evaluate it.
Whether you are using the A-model, or one of the other models of showing training effectiveness (Kirkpatrick, Phillips, Bersin etc.), you need good data to be able to show effectiveness. How and where do you get this data?
A neat concept in the A-model is the evaluation cube shown below, which identifies 3x3x3 different ways in which you can collect data to feed into the A-model.
The evaluation cube shows 3 dimensions of possible evaluation
You can evaluate:
- Problem – the business problem that you’re seeking to address (e.g. increase sales or improve productivity)
- Performance – people’s performance in job roles (e.g. how someone’s capability changes or improves)
- Program – the training or learning intervention and how successful it is (how a specific intervention acts)
With data from:
- The individual (e.g. tests or surveys taken by people to measure knowledge, skills, performance or attitudes)
- Observers (assessments where someone rates another, for example an observational or workplace assessment of skills, attitude or performance)
- Systems (gathering data from records. e.g. sales or accounting or HR records or other systems
And you can collect data relative to:
- Self – how someone’s performance e.g. on a test changes over time, for instance from pre-training to post-training or from one year to another
- Others or the norm – how data about a person or group compares to the distribution of others, for example how this department or person compares to other departments or people, or the norm for the organization
- Criteria – how data compares against targets or benchmarks, absolute measures
For example, if you are looking to identify data about Performance, you could consider 9 facets of the cube: individual/observers/systems vs self/others/criteria. And you might use an observational assessment before and after the learning intervention (observers/self) to identify change and also a test of an aspect of performance after the event (individual/others) to measure capability compared to the norm for the organization.
Thinking about the different facets of the cube can be a good way to brainstorm ideas for assessments and data gathering.
I hope these ideas spur you to effective measurement of training interventions, whatever methodology you are using.
For more on the A-model, see Dr. Bruce C. Aaron’s excellent white paper, available here (free with registration).