6 Stages to Getting Assessment Results You Can Trust: White Paper

Headshot JuliePosted by Julie Delazyn

Modern organizations need their people to be competent.

Would you be comfortable in a high-rise building designed by an unqualified architect? Would you fly in a plane whose pilot hadn’t passed a flying test? Would you send a sales person out on a call  if they didn’t know what your products do? Can you demonstrate to a regulatory authority that your staff are competent and fit for their jobs if you do not have trustworthy assessments?

In all these cases and many more, it’s essential to have a reliable and valid test of competence. If you do not ensure that your workforce is qualified and competent, then you should not be surprised if your employees have accidents, cause your organization to be fined for regulatory infractions, give poor customer service or can’t repair systems effectively.

The white paper, Assessment Results you Can Trust, explains that trustworthy assessment results must be both valid (measuring what you are looking for them to measure) and reliable (consistently measuring what you want to be measured).

For assessments to be valid and reliable, it’s necessary to follow structured processes at each step from planning through authoring to delivery and reporting.

The white paper covers these six stages of the assessment process:

  • Planning assessment
  • Authoring items
  • Assembling assessment
  • Pilot and review
  • Delivery
  • Analyze results

Following the advice in the white paper and using the capabilities it describes will help you produce assessments that are more valid and reliable — and hence more trustable.

To download the complimentary white paper, click here.


Can you be GDPR compliant without testing your employees?

Posted by John Kleeman

The GDPR is a new extra-territorial, data protection law which imposes obligations on anyone who processes personal data on European residents. It impacts companies with employees in Europe, awarding bodies and test publishers who test candidates in Europe, universities and colleges with students in Europe and many others. Many North American and other non-European organizations will need to comply.

See my earlier post How to use assessments for GDPR compliance for an introduction to GDPR. The question this blog post addresses is whether it’s practical for a large organization to be compliant with the GDPR without giving tests and assessments to their employees?

I’d argue that for most organizations with 100s or 1000s of employees, you will need to test your employees on your policies and procedures for data protection and the GDPR. Putting it simply, if you don’t and your people make mistakes, fines are likely to be higher.

Here are four things the GDPR law says (I’ve paraphrased the language and linked to the full text for those interested):


1. Organizations must take steps to ensure that everyone who works for them only processes personal data based on proper instructions. (Article 32.4)

2. Organizations must conduct awareness-raising and training of staff who process personal data (Article 39.1). This is extended to include “monitoring training” for some organizations in Article 47.2.

3. Organizations must put in place risk-based security measures to ensure confidentiality and integrity and must regularly test, assess and evaluate the effectiveness of these measures. (Article 32.1)

4. If you don’t follow the rules, you could be fined up to 20 million Euros or 4% of turnover. How well you’ve implemented the measures in article 32 (i.e. including those above) will impact how big these fines might be. (Article 83.2d)


So let’s join up the dots.

Firstly, a large company has to ensure that everyone who works for it only processes data based on proper instructions. Since the nature of personal data, processing and instructions each have particular meanings, this needs training to help people understand. You could just train and not test, but given that the concepts are not simple, it would seem sensible to test or otherwise check their understanding.

A company is required to train its employees under Article 39. But the requirement in Article 32 is for most companies stronger. For most large organizations the risk of employees making mistakes and the risk of insider threat to confidentiality and integrity is considerable. So you have to put in place training and other security measures to reduce this risk. Given that you have to regularly assess and evaluate the effectiveness of these measures, it seems hard to envisage an efficient way of doing this without testing your personnel. Delivering regular online tests or quizzes to your employees is the obvious way to check that training has been effective and your people know, understand and can apply your processes and procedures.

Lastly, imagine your company makes a mistake and one of your employees causes a breach of personal data or commits another infraction under the GDPR? How are you going to show that you took all the steps you could to minimize the risk? An obvious question is whether you did your best to train that employee in good practice and in your processes and procedures? If you didn’t train, it’s hard to argue that you took the proper steps to be compliant. But even if you trained, a regulator will ask you how you are evaluating the effectiveness of your training. As a regulator in another context has stated:

“”where staff understanding has not been tested, it is hard for firms to judge how well the relevant training has been absorbed”

So yes, you can imagine a way in which a large company might manage to be compliant with the GDPR without testing employees. There are other ways of checking understanding, for example 1:1 interviews, but they are very time consuming and hard to roll out in time for May 2018. Or you may be lucky and have personnel who don’t make mistakes! But for most of us, testing our employees on knowledge of our processes and procedures under the GDPR will be wise.

Questionmark OnDemand is a trustable, easy to use and easy to deploy system for creating and delivering compliance tests and assessments to your personnel. For more information on using assessments to help ensure GDPR compliance visit this page of our website or register for our upcoming webinar on 29 June.

Reminiscing about Santa Fe: Presentations, pictures & the weird and wonderful art house

Posted by Chloe Mendonca

After eagerly looking forward to Questionmark’s most important annual learning event for months, it was over before we even knew it! The Questionmark Conference gave all of us three special days to meet so many of our globally dispersed customers and employees face to face, learn best practices, have fun with one another and discuss new ways to leverage Questionmark’s technologies.

This year I was fortunate enough to be there, and a big highlight was getting a deeper understanding of how others are using Questionmark’s technologies. From our evening networking events to our stimulating panel discussion — which brought together experts from the US State Department, Caterpillar Inc., Scantron and Compass Consultants to discuss best practices for making data work within learning and assessment programs — to more specific breakout sessions, our guest speakers did a wonderful job of sharing lessons learned and best-practice tips.

Todd Horner from Accenture, for example, hosted a great discussion, “Taking the Migraine out of Migration: Accenture’s journey to next-gen authoring.” He spoke about the shared “fear of the unknown” and how to get around change-management challenges. Lauri Buckley and Lindsey Clayton from Caterpillar Inc, delivered an impressive presentation, “A Process to Mastery: Assessments as career development tools,” during which they shared valuable tips about how to effectively design and develop various types of competence assessments, from proficiency tests to validation and observational assessments. You can get the handouts from these presentations and more right here.

For those who couldn’t be there in person, we webcast selected conference sessions — hitting record numbers online. If you joined us for the webcast, got a sense of the Questionmark Conference atmosphere and want to join us in person next year, keep your eyes peeled for our dates and location announcement coming to the blog in the next few months. See the recordings of our selected webcast sessions at: www.questionmark.com/go/2017uconwebcast (Please note you must be logged into the website with your Questionmark username and password).

I’d like to take this opportunity to say a big thank you to all of our wonderful speakers for taking the time to share their knowledge. Without them there would be no conference!

Now for the bit you’ve all been waiting for… conference pictures! To all those who went back to the office struggling to describe the weird and wonderful art house that is Meow Wolf’s House of Eternal Return, hopefully these snaps will make things a little easier 😊  View conference and evening event pictures here on our flickr page.

What did you enjoy most about Questionmark Conference 2017? Leave me a comment below and stay in touch!


Just in case you missed it…

John Kleeman, Questionmark’s Founder & Executive Director reported back 6 good practice tips heard in Santa Fe.

7 ways assessments can save you money and protect your reputation [Compliance webinar]

Julie ProfilePosted by Julie Delazyn

Last week, illegal banking practices cost Wells Fargo, one of America’s largest banks, $185 million in fines. Regulators have called the scandal “outrageous” and stated that the widespread nature of the illegal behavior shows the bank lacked the necessary controls and oversight of its employees.

Educating and monitoring employee understanding of proper practices is vital for regulatory compliance.  How do you ensure your workers are compliant with the rules and regulations in your industry? How do you prove that employee training is understood?

Register today for the FREE webinar: 7 Ways Assessments Fortify Compliance

The webinar will examine real-world examples of how assessments are used to strengthen compliance programs. It will also provide tips for developing valid, reliable assessments.

Predicting Success at Entel Begins with Trustworthy Assessment Results [Case Study]

Julie ProfilePosted by Julie Delazyn

Entel is one of Chile’s largest telecommunications firms, serving both the consumer and commercial sectors. With more than 12,000 employees across its extended Entelenterprise, Entel provides a broad range of mobile and fixed communications services, IT and call center outsourcing and network infrastructure services.

The Challenge

Success in the highly competitive mobile and telecommunications market takes more than world-class infrastructure, great connectivity, an established dealer network and extensive range of retail location. Achieving optimal on-the-job performance yields a competitive edge in the form of satisfied customers, increased revenues and lower costs. Yet actually accomplishing this objective is no small feat – especially for an industry job role notorious for high turnover rates.

With these challenges in mind, Entel embarked on an innovative new strategy to enhance the predictability of the hiring, onboarding, training and developing practices for its nationwide team of 6,000+ retail store and call center representatives.

Certification as a Predictive Metric

Entel conducted an exhaustive analysis – a “big data” initiative that mapped correlations between dozens of disparate data points mined from various business systems, HR systems as well as assessment results – to develop a comprehensive model of the factors contributing to employee performance.

Working with Questionmark OnDemand enabled Entel to create the valid and reliable tests and exams necessary to measure and document representatives’ knowledge, skills and abilities.

Find out more about Entel’s program planning and development, which helped identify and set benchmarks for required knowledge and skills, optimal behaviors and performance metrics, its use of SAP SuccessFactors to manage and monitor performance against many of the key behavioral aspects of the program, as well as the growing role their trustworthy assessment results are having on future product launches and the business as a whole.

Click here to read the full case study.

6 Steps to Authoring Trustworthy Assessments

AprilPosted by April Barnum

I recently met with customers and the topic of authoring trustworthy assessments and getting back trustable results was a top concern. No matter what they were assessing on, everyone wants results that are trustable, meaning that they are both valid and reliable. The reasons were similar, with the top three being: Safety concerns, being able to assert job competency, and regulatory compliance. I often share this white paper: 5 steps to better tests, as a strong resource to help you plan a strong assessment, and I encourage you to check it out. But here are six authoring steps to that can help you achieve trustworthy assessment results:

  1. Planning the assessment or blueprinting it. You basically are working out what it is that the test covers.
  2. Authoring or creating the items.
  3. Assembling the assessment or harvesting the items and assemble them for use in a test.
  4. Piloting and reviewing the assessment prior to using it for production use.
  5. Delivering the assessment or making the assessment available to participants; following security, proctoring and other requirements set out in the planning stage.
  6. Analyzing the results of the assessment or looking at the results and sharing them with stakeholders. This step also involves using the data to weed out any problem items or other issues that might be uncovered.

Each step contributes to the next, and useful analysis of the results is only possible if every previous stage has been done effectively. In future posts, I will go into each step in detail and highlight aspects you should be considering at each stage of the process.

assessment plan

Next Page »