Authoring compliance-related assessments: good practice recommendations

Headshot JuliePosted by Julie Delazyn

Last week I wrote about deploying compliance-related assessments, as part of a series of posts offering good practice recommendations from our white paper, The Role of Assessments in Mitigating Risk for Financial Services Organizations.

This paper describes five stages of deploying legally defensible assessments, along with specific recommendations for people in different job roles. Some of these recommendations are specific to Questionmark technologies, but most can be applied to any testing and assessment system.

The five stages:

Compliance five steps
Today, let’s look at good practice for the third stage: authoring. You will find more recommendations in the White Paper:

authoring chart

Sharon Shrock & William Coscarelli’s Criterion-Referenced Test Development: Technical and Legal Guidelines for Corporate Training provides actionable, practical advice on test development. Sharon and Bill will conduct a workshop on writing valid, reliable tests in Baltimore on Sunday, March 3. Participants will explore testing best practices and will learn how to meet rigorous competency testing standards.

You can register for this workshop when you register for the Questionmark Users Conference or add the workshop later. It’s up to you!

Ten assessment types that can help mitigate risk

Posted by Julie Delazyn

Mitigating risk –- most notably the risk of non-compliance — is a key component of success in the financial services industry. Other risks abound, too, such as losing customers and/or good employees.

If employees don’t understand and follow the processes that organizations put in place to mitigate risk and maintain compliance, the risk of non-compliance increases – and a business is less likely to succeed.

Online assessments do a lot to help ensure that employees know the right procedures and follow them. Here are 10 assessment types that play essential roles here:

(1) Internal exams -– check your employees are competent

Some companies administer internal competency exams annually –- and do so more frequently. It’s also good to give these exams when regulations changed and new products are introduced. These exams address compliance with competency requirements and at the same time help employees prove they know how to do their jobs.

(2) Knowledge checks – confirm learning and document understanding

Running knowledge checks or post-course tests (also called Level 2s) right after training helps you find out whether the training has been understood. This also helps reduce forgetting.

(3) Needs analysis / diagnostic tests – allow testing out

These tests, which measure current skills and knowledge about particular topics, can be used as training needs assessments and/or pre-requisites for training. And if someone already has the critical skills and knowledge, he or she can “test out” and avoid unnecessary and costly training.

(4) Observational assessments – measure skills via role plays, customer visits

When checking practical skills, it’s common to have an observer monitor an employee to see if they are following correct procedures. With so many people using smartphones and tablets, such as the Apple iPad, it’s viable to use a mobile device for these assessments — which are great for measuring behavior, not just knowledge.

(5) Course evaluation surveys

These surveys, also called “level 1” or “smile sheet” surveys, let you check employee reaction following training. They are a key step in evaluating training effectiveness. You can also use them to gather qualitative information on topics, such as how well policies are applied in the field. Here is an example fragment from a course evaluation survey:

(6) Employee attitude surveys

Employee attitude surveys ask questions of your workforce or sections of it. HR department often use them to measure employee satisfaction, but they also can be used in corporate compliance to determine attitudes about ethical and cultural issues.

(7) Job task analysis surveys –- to fairly identify tasks against which to check compliance

How do you know that your competency assessments are valid and that they are addressing what is really needed for competence in a job role? A job task analysis (JTA) survey asks people who are experts in a job how important the task is for the job role and how often it is done. Analysis of JTA data lets you weight the number of questions associated with topics and tasks so that a competency test fairly measures the importance of different elements of a job role. Here is an extract from a typical JTA23:

(8) Practice tests

These often use questions that are retired from the exam question pool but remain valid. Candidates can take practice tests to assess their study needs and/or gain candidates experience with the technology and user interface before they take a real exam. This helps to reduce exam anxiety, and it’s important for less computer-literate candidates. Practice tests are also helpful when deploying new exam delivery technology.

(9) Formative quizzes during learning -– to help learning

These quizzes are those we are all familiar with: short quizzes during learning to inform instructors and learners about whether learners have understood the learning or need deeper instruction. Such quizzes can also diagnose misconceptions and also help reduce forgetting.

(10) 360-degree assessments of employees

A 360-degree assessment solicits opinions about an employee’s competencies from his/her superiors, reports and peers. It will usually cover job-specific competencies and general competencies such as integrity and communication skills. In compliance, such surveys allow you to potentially identify issues in people’s behavior and competencies that need review.

For more in-depth coverage of this subject, read our white paper,  The Role of Assessments in Mitigating Risk for Financial Services Organizations, which you can download free after login or sign-up.

UK Webinar: Using Assessments to Mitigate Risk and Ensure Regulatory Compliance

Posted by Chloe Mendonca

Recent news stories about regulatory compliance in the financial services industry highlight the need for effective training and assessment in this sector.

In response, we’re offering a webinar that highlights the role assessments in mitigating risk and supporting compliance.

Join us at 11 a.m. British Summer Time (London — UTC +1) on Tuesday, 3rd July for Using Assessments to Mitigate Risk and Ensure Regulatory Compliance

In addition to discussing the vital role of assessments in mitigating risk, this webinar will share how Questionmark assessment technologies can be used to support and promote compliance.

We will cover these and other topics:

  • Demonstrating your organisation’s commitment to comply with laws, making prosecution less likely if an individual mis-steps
  • How assessments can warn of a lack of knowledge or understanding before these impact the business
  • Providing effective documentation that training has taken place and that employees understand what they are required to do in line with regulations
  • Saving time by allowing knowledgeable employees to “test out” of training on topics in which they are already expert

Click here to register

The Role of Assessments in Mitigating Risk for Financial Services Organizations

Posted by John Kleeman

Ever since Questionmark started in the 1980s, one of our largest areas of use has been the financial services industry.   Mitigating risk is key component of success for banks, insurance companies and others in financial services. Assessments play a crucial role here, and many of the world’s leading financial services companies use Questionmark assessments to help mitigate risk.

With colleagues, I’ve written a white paper on The Role of Assessments in Mitigating Risk for Financial Services Organizations. You can download the white paper, it’s free with registration. I’d like to thank several readers of this blog for their review and input into draft versions of the white paper.

Mitigating risk is key component of success in the financial services industry. The risk of non-compliance is an obvious concern, but other risks abound, too – such as losing customers and/or good employees.

Every organization defines processes to mitigate risk and maintain compliance, and it’s essential for employees to understand them and follow these processes. If they don’t, the risk of non-compliance increases. Training certainly helps, but without follow-up assessments there’s no proof that people have learned what they need to know. And organizations that can’t prove they have taken adequate steps to educate employees face higher non-compliance penalties.

How can organizations ensure that employees have demonstrated competence in matters of regulatory compliance?

Assessments play an essential role: Quizzes, tests and surveys are the most cost-effective way to confirm and document mandatory regulatory training. Whether required by regulators or not, assessments delivered online to employees are one of the few ways — and likely the best way — to touch individually on your entire workforce and ensure that people understand their roles in your business and what is required of them to meet regulatory and business needs. Effective assessments improve performance – making for happier customers and employees – while at the same time aiding compliance.

Centrally managed assessments delivered via IT technology bring both peace of mind and tremendous business value by:

  • helping employees develop the knowledge skills and attitudes to perform well
  • ensuring that employees act competently and follow proper procedures
  • demonstrating your organization’s commitment to complying with laws, making prosecution less likely if an individual missteps
  • providing the most conclusive and effective documentation that training has taken place and that employees understand what they are required to do
  • warning of a lack of knowledge or understanding before these impact the business
  • highlighting training needs so you can direct resources towards areas of weakness
  • saving time by allowing knowledgeable employees to “test out” of training on topics they are already expert in
  • allowing you to turn a compliance program that simply focuses on regulations into one that helps your employees improve customer service

Online assessments have long been used in compliance, but now more than ever. Most large companies today see online assessments as a critical part of their compliance programs. The paper explains why, by describing today’s regulatory landscape and showing the business benefits of using assessments to meet its demands. It also describes specific ways to use assessments within a compliance program, examines security and accessibility issues and offers role-specific best practice guidance for implementing a legally defensible assessment program.

I hope the paper proves useful reading!