What is the most important thing about a compliance assessment?

John Kleeman HeadshotPosted by John Kleeman

What is the most important thing about a compliance assessment? Almost certainly that it is  reliable and valid.

An assessment is reliable when it works consistently. An assessment is considered reliable if we get the same result repeatedly. If an assessment isn’t reliable, then it doesn’t normally have much use.

An assessment is valid if it measures what it is supposed to measure. In compliance, a valid assessment often measures the specific knowledge and skills that were intended to be measured to check competence. More broadly, if a survey is administered to happy people, the results should show that they’re all happy. Similarly if a group of people who are all knowledgeable are tested, the test results should reveal that they’re all knowledgeable. Good assessments are both reliable and valid.

A good way of thinking about whether assessments are reliable and valid is to think about throwing darts at a dart board.

Reliable but not valid

In the diagram below, all the darts are stuck in the same area, illustrating that the thrower—the analogue of an assessment—is reliable and consistent, but unfortunately his throws are not valid. If his throws were valid, all the darts would be in the center, the bulls-eye.

Reliable but not valid

Not reliable

In the diagram below, the darts have landed all over the board. This assessment is not reliable because it’s not consistent.  An assessment can be reliable without being valid, but it’s not possible for it to be valid if it’s not reliable.

Not reliable

 Valid and reliable

Finally, the last example is of an assessment that is both reliable and valid, because all of the scores are clustered together and on target.

Valid and reliable

 I hope this explanation of reliability and validity is useful as an introduction or reminder.

You can click here if you would like information about using Questionmark assessments for regulatory compliance, including links to case studies and white papers. And if you are attending the Questionmark European Users Conference in Barcelona next week, you are welcome to attend the presentation I am giving there about 7 good reasons to use online assessments for regulatory compliance.

Leave a Reply