Will testing employees reduce fines for compliance errors?

John Kleeman Headshot

Posted by John Kleeman

If a bank faces a fine of millions for money laundering and then can prove, defensibly, that the ‘accused’ had passed competency tests, would that reduce or eliminate the fine? More generally, suppose employees do something wrong and the corporation is facing a regulatory fine. Does it make a difference if those employees were certified? Is it a defence against regulatory action that you took all the measures you could to prevent error?

We are asked this question from time to time, and the answer varies considerably by regulator and by offence. But in general having competent/certified people and good compliant processes will reduce the impact to the corporation of making a compliance mistake. In some cases it might eliminate a fine, but usually not.

Here are three specific examples where a good compliance program can reduce or eliminate fines.

Prosecutors should therefore attempt to determine whether a corporation’s compliance program is merely a “paper program” or whether it was designed, implemented, reviewed … in an effective manner. In addition, prosecutors should determine whether the corporation has provided for a staff sufficient to audit, document, analyze, and utilize the results of the corporation’s compliance efforts. Prosecutors also should determine whether the corporation’s employees are adequately informed about the compliance program and are convinced of the corporation’s commitment to it. This will enable the prosecutor to make an informed decision as to whether the corporation has adopted and implemented a truly effective compliance program that … may result in a decision to charge only the corporation’s employees and agents or to mitigate charges or sanctions against the corporation.

  • The UK Ministry of Justice guidance on the Bribery Act recommends communication and training around bribery and says that “it is a full defence for an organisation to prove that despite a particular case of bribery it nevertheless had adequate procedures in place to prevent persons associated with it from bribing.”
  • Similarly, in Spain, the Spanish criminal code has been updated so that companies may avoid criminal prosecution if they have an effective compliance program in effect including evidence that employees have had sufficient training in the compliance program.

Fines rising to over one billion pounds in 2014 and nearly one billion pounds in 2015In general, the issue is more diffuse. For example, the UK Financial Conduct Authority, which has issued many huge fines over the years (see graph right), does not seem to explicitly reduce fines based on compliance measures.

But its Penalties Manual does say that fines should be increased if the actions are deliberate or reckless or if the breach resulted from systematic weaknesses in the firm’s procedures. Equally, if the breach was inadvertent and there is no evidence that the breach indicates a widespread problem or weakness, the fine might be lower.

So how best to summarize this?

The biggest benefit of a programme for competency testing for employees is that, in conjunction with other compliance measures, it will reduce the chances of an infraction in the first place.

Having certified or competent people is not a “get out of jail free” card but if part of a professional compliance programme, it will help with many regulators in mitigating financial penalties after an infraction.

How much do you know about defensible assessments?

Julie Delazyn HeadshotPosted by Julie Delazyn

This quiz is a re-post from a very popular blog entry published by John Kleeman.

Readers told us that it was instructive and engaging to take quizzes on using assessments, and we like to listen to you! So here is the second quiz in a pre-published series of quizzes on assessment topics. This one was authored in conjunction with Neil Bachelor of Pure Questions. You can see the first quiz on Cut Scores here.

As always, we regard resources like this quiz as a way of contributing to the ongoing process of learning about assessment. In that spirit, please enjoy the quiz below and feel free to comment if you have any suggestions to improve the questions or the feedback.

Is a longer test likely to be more defensible than a shorter one? Take the quiz and find out. Be sure to look for your feedback after you have completed it!

NOTE: Some people commented on the first quiz that they were surprised to lose marks for getting questions wrong. This quiz uses True/False questions and it is easy to guess at answers, so we’ve set it to subtract a point for each question you get wrong, to illustrate that this is possible. Negative scoring like this encourages you to answer “Don’t Know” rather than guess; this is particularly helpful in diagnostic tests where you want participants to be as honest as possible about what they do or don’t think they know.

What’s the best way to document that training has taken place?

Posted by John Kleeman

I thought readers of this blog might enjoy seeing a copy of my article recently published in Compliance & Ethics Professional Magazine July/August 2012. You can see a reprint of the article here.

Compliance & Ethics Professional is a journal published by the non-profit SCCE (Society of Corporate Compliance and Ethics) which is dedicated to improving the quality of corporate governance, compliance and ethics. Their website at www.corporatecompliance.org has lots of compliance resources.Compliance & Ethics Professional

The article suggests there are three ways to document training:

  • Have the instructor take attendance
  • Get employees signatures confirming they have understood the training
  • Administer a test after training so that employees can demonstrate understanding

Taking attendance is simple and easy to do, but it documents attendance and not understanding. It’s common for people to start training and drift off, either in person or just in their head. Taking attendance doesn’t help here.

Getting people to sign that they understood the training takes things one step further, but in many corporate cultures, people may well sign just to get the compliance requirement out of the way, without fully understanding the training. And even if they mean well, thinking you know something and actually knowing it are not always the same.

Giving a test to confirm understanding of training takes a bit more effort. But it has the huge advantage that you are not just checking that the employee was present in the training. Nor are you just checking that the employee thought or claimed they understood the training. You are getting firm evidence that the training was understood.

The article argues that giving a test after training is the best way to document training, and much more reliable for compliance purposes than the other two methods. If you are interested in reading the full article, it’s available here.

What is the Angoff Method?

Posted by Julie Delazyn

When creating tests that define levels of competency as they relate to performance, it’s essential to use a reliable method for establishing defensible pass/fail scores.

One of these is the Angoff Method, which uses a focus-group approach for this process. This method has a strong track record and is widely accepted by testing professionals and courts.

Subject-matter experts (SMEs) review each test question and then predict how many minimally-qualified candidates would answer the item correctly. The average of the judges’ predictions for test questions is used to calculate the passing percentage (cut score) for a test.

Basing cut scores on empirical data instead of choosing arbitrary passing scores helps test developers produce legally defensible tests that meet the Standards for Educational and Psychological Testing. The Angoff Method offers a practical way to achieve this.

View this SlideShare presentation to learn more:

The Future Looks Bright

Posted by Jim Farrell

Snapshot from a “Future Solutions” focus group

Our Users Conferences are a time for us to celebrate our accomplishments and look forward to the challenges that lie in front of us. This year’s conference was full of amazing sessions presented by Questionmark staff and customers. Our software is being used to solve complex business problems. From a product standpoint, it is the very exciting to bring these real-life scenarios to our development teams to inspire them.

So where do we go from here? The Conference is our chance to stand in front of our customers and get feedback on our roadmap. We also held smaller “Future Solutions” focus groups to get feedback from our customers on what we have done and what we could do in the future to help them. In the best of times, these are an affirmation that we are on the right path. This was definitely one of those years.

One of our Future Solutions sessions focused on authoring. During that session, Doug Peterson and I laid out the future of Questionmark Live. This included an aggressive delivery cycle that will bring future releases at a rapid pace. Stay tuned for videos on new features available soon.

Ok…enough about us. This conference is really about our customers. The panel and peer discussion strand of this year’s conference had some of the most interesting topics. John Kleeman has already mentioned the security panel with our friends from Pearson Vue, ProctorU, Innovative Exams and Shenandoah University.

Another session that stood out was as a peer discussion test defensibility using the Angoff method to set cut scores. This conversation was very  interesting to me as someone who once had to create defensible assessments. I am eager to see organizations utilize Angoff because not only do  you want legally defensible assessments, you want to define levels of competency for a role and be able to determine how that can  predict future performance.

For those of you who do not know, the Angoff method is a way for Subject Matter Experts (SMEs) to grade the probability of a marginal student getting a question right. Attendees at this conference session were provided a handout that includes a seven-step flowchart guiding them in the design, development and implementation of the Angoff method.

If you are interested in Angoff and setting test scores I highly recommend reading Criterion-Referenced Test Development written by our good friends Sharon Shrock and Bill Coscarelli.

We really hope to see everyone at the 2013 Users Conference in Baltimore March 3 – 6. (I am hoping we may even get a chance to visit the beautiful Camden Yards!)

Proving compliance – not just attendance

Posted by John Kleeman

Many regulators require you to train employees – in financial services, pharmaceuticals, utilities and in health & safety across all industries. You need to train them and when you are audited or if something goes wrong, you need to document that you did the training. To quote the US regulator OSHA: Documentation can also supply an answer to one of the first questions an accident investigator will ask: “Was the injured employee trained to do the job?”

Is it good enough to get the participant to sign something saying that they’ve attended the training or read the safety manual? An excellent blog series on the SafetyXChange says no:

Some companies ask their workers to sign a form after training sessions acknowledging that they understood the lesson and will put it into practice. Don’t let these forms lull you into a false sense of security. “Most workers will just sign these things without even reading them, let alone making sure that they understood everything you told them,” says a health and safety attorney in New York City. This is especially true if the training and instructions are complicated.

In the safety field, a US Appeals Court law case ruled in 2005 (my underlining):

Merely having an individual sign a form acknowledging his responsibility to read the safety manual is insufficient to insure that the detailed instructions contained therein have actually been communicated.

Two good ways to show that someone not only attending the training but also understood it:

Workplace assessment on ladder use

Give employees a test or quiz at the end of the training to confirm that they understood it. This will also give them practice retrieving information to slow the forgetting curve (see Answering Questions directly helps you learn). And it will allow you to pick out people who didn’t get the learning or weak points in the class.

For more practical skills, you might want to observe people to check they understood the training and can practice it, or in the safety world demonstrate that they can do the job safely. For example the screenshot on the right shows how a supervisor can use an iPad to check and log someone’s skill on using a ladder.

My view is that you want to give these kinds of tests for two reasons. First and most importantly, you want to prevent your employees from falling off ladders or making other mistakes. Second, if something does go awry, you want evidence that you’ve trained people well.

If you’re interested in this area you might check out a recent, busy discussion (registration required) on the LinkedIn Compliance Exchange forum. Paraphrasing some of the views there:

Yes, you should give a quiz as it proves attendance – videoing the training is another option.

Yes, you should give a test and regulators in particular the US FDIC are increasingly demanding this

No. Danger of a test is that you need to take action if scores are bad, which may give you a lot of work. Safer not to ask the questions in case you don’t like the answers.

Yes, you should give a test but it can be a very easy and simple one, to check basic understanding and prove attendance

Yes, you should test, as well as confirming understanding it will also highlight vulnerabilities in the training

What do you think? Use the reply form below and contribute to the dialog.