Do privacy laws mean you have to delete a test result if a test-taker asks you to?

Posted by John Kleeman

We have all heard about the “right to be forgotten”, which allows individuals to ask search engines or other organizations to delete their personal data. This right was made stronger in Europe in 2018, when the General Data Protection Regulation (“GDPR”) entered into force, and is gradually becoming recognized in some form in other jurisdictions, for example in the new California privacy law, the California Consumer Privacy Act (“CCPA”).

I’m often asked questions by customers about what the situation is if test-takers ask to delete the results for tests and exams.  Let’s take an example:

  • Your organization runs a global certification program for third party candidates;
  • One of your European candidates takes an exam in your program;
  • The candidate then reaches out to you and asks for all their personal data to be deleted.

What do you need to do? Do you verify his/her identity and delete the data? Or can you hold onto it and deny the request if you have reasons why you need to – for example, if you want to enforce retake policies or you are concerned about possible cheating. Here is an answer based on typical circumstances in Europe (but please get advice from your lawyer and/or privacy adviser regarding your specific circumstances).

Under the GDPR, although as a general principle you do need to delete personal data if retaining it for a longer period cannot be justified for the purposes for which it was initially collected or another permitted lawful purpose, there are exemptions which may allow you to decline an erasure request.

For example, you may refuse to delete personal data in response to a request from an individual if retaining the data is necessary to establish, exercise or defend against legal claims. If you follow this exception, you must be comfortable that retention of the data is necessary, and you must only use the data for this purpose, but you do not need to fully delete it.

Another broader reason for refusing to delete data may arise if you articulate in advance of the candidate taking the exam that processing is performed based on the data controller’s (usually the test sponsor’s) legitimate interests. The GDPR permits processing based on legitimate interests if you balance such interests against the interests, rights and freedoms of an individual. The GDPR also specifically says that such legitimate interests may be used to prevent fraud (and this obviously includes test fraud).

If you want to be able to refuse to delete information on this basis:

  • You should first conduct and document a legitimate interests assessment which justifies the purpose of the processing, considers whether the processing is really necessary, and balances this against the individual’s interests. (See this guidance from the UK Information Commissioner for more information);
  • You should communicate to candidates in advance, for example in your privacy policy or candidate agreement, that you are processing their data based on explained legitimate interests;
  • If you then later receive a deletion request, you should carefully analyse whether notwithstanding the request you have overriding legitimate interests to retain the data;
  • If you conclude that you do have such an interest, you should only retain the data for as long as that continues to be the case and only keep the data to which the overriding legitimate interest applies. This might mean that you have to delete some data, but can keep the rest.
  • You also need to let the individual know about your decision promptly providing them with information including their right to complain to the appropriate supervisory authority if they are unhappy with your decision.

The CCPA also has some exceptions where you do not need to delete data, including where you need to retain the data to prevent fraudulent activity.

In general, you may well want to follow delete requests, but if you have good reason not to, you may not need to.

For further information, there is some useful background in the Association of Test Publishers (ATP) GDPR Compliance Guide, in other ATP publications and in Questionmark’s white paper “Responsibilities of a Data Controller When Assessing Knowledge, Skills and Abilities” obtainable at https://www.questionmark.com/wc/WP-ENUS-Data-Controller.

I hope this article helps you if this issue arises for you.

NEW: Listen Now to “Unlocking the Potential of Assessments” Podcast

Posted by Kristin Bernor

Welcome to Questionmark’s new podcast series, “Unlocking the Potential of Assessments.” This monthly series delves into creating, delivering and reporting on valid and reliable assessments.

“Unlocking the Potential of Assessments” will offer advice and thought leadership to those just starting out with assessments, those who have been in the industry at length and anyone with a keen interest in the future of assessments lead by host, John Kleeman – Questionmark’s Founder and Executive Director.

In our first episode, John spoke with assessment luminary, Jim Parry, Owner and Chief Executive Manager of Compass Consultants. Jim has over 40 years experience as a course designer, developer and instructor. He served over 22 years with the United States Coast Guard and was employed for nearly 12 years by the US Coast Guard as a civilian employee as the Test Development and e-testing Manager at a major training command.

During his tenure, Jim guided the move from paper to online testing for the entire Coast Guard and developed the first ever Standard Operating Procedure, a document of over 300 pages, which established policy and guidelines for all testing within the Coast Guard. He is a consulting partner with Questionmark and has presented numerous best practice webinars.

Subscribe to the podcast series today and join Questionmark on our quest to discover and examine the latest in best practice guidance with a wide array of guests – including assessment luminaries, industry influencers, SMEs and customers – and discuss “all things assessment.”

Don’t miss out. For our next episode, John will be speaking with our very own Steve Lay, Questionmark’s product manager and an expert on scalable, computerized assessment and integration between systems. Subscribe today so you don’t miss it.

You can subscribe to the series by visiting our podcast page and selecting your preferred player.

Please reach out to me with any suggestions of further topics you’d like explored or assessment luminaries you want to hear from.

10 Reasons Why Frequent Testing Makes Sense

Posted by John Kleeman

It matters to society, organizations and individuals that test results are trustable. Tests and exams are used to make important decisions about people and each failure of test security reduces that trustworthiness.

There are several risks to test security, but two important ones are identity fraud and getting help from others. With identity fraud, someone asks a friend to take the test for them or pays a professional cheater to take the test and pretend to be them. With getting help from others, a test-taker subverts the process and gets a friend or expert to help them with the test, feeding them the right answers. In both cases, this makes the individual test result meaningless and detracts from the value and trustworthiness of the whole assessment process.

There are lots of mitigations to these risks – checking identity carefully, having well trained proctors, using forensics or other reports and using technical solutions like secure browsers – and these are very helpful. But testing more frequently can also reduce the risk: let me explain.

Suppose you just need to pass a single exam to get an important career step – certification, qualification or other important job requirement, then the incentive to cheat on that one test is large. But if you have a series of smaller tests over a period, then it’s more hassle for a test taker to conduct identity fraud or to get help from others each time. He or she would have to pay the proxy test taker several times.  And make sure the same person is available in case photos are captured. And for the expert help you also must reach out more often, and evade whatever security there is each time

There are other benefits too; here is a list of ten reasons why more frequent testing makes sense:

  1. More reliable. More frequent testing contributes to more reliable testing. A single large test is vulnerable to measurement error if a test taker is sick or has an off day, whereas that is less likely to impact frequent tests.
  2. More up to date. With technology and society changing rapidly, more frequent tests can make tests more current. For instance, some IT certification providers create “delta” tests measuring understanding of their latest releases and encourage people to take quarterly tests to ensure they remain up to date.
  3. Less test anxiety. Test anxiety can be a big challenge to some test takers (see Ten tips on reducing test anxiety for online test-takers), and more frequent tests means less is at stake for each one, and so may help test takers be less anxious.
  4. More feedback. More frequent tests give feedback to test takers on how well they are performing and allow them to identify training or continuing education to improve.
  5. More data for testing organization. In today’s world of business intelligence and analytics, there is potential for correlations and other valuable insight from the data of people’s performance in a series of tests over time.
  6. Encourages test takers to target retention of learning. We all know of people who cram for an exam and then forget it afterwards. More frequent tests encourage people to plan to learn for the longer term.
  7. Encourages spaced out learning. There is strong evidence that learning at spaced out intervals makes it more likely knowledge and skills will be retained. Periodic tests encourage revision at regular intervals and so make it more likely that learning will be remembered.
  8. Testing effect. There is also evidence that tests themselves give retrieval practice and aid retention and more frequent tests will give more such practice.
  9. More practical. With online assessment software and online proctoring, it’s very practical to test frequently, and no longer necessary to bring test takers to a central testing center for one off large tests.
  10. Harder to cheat. Finally, as described above, more frequent testing makes it harder to use identity fraud or to get help from others, which reduce cheating.

I think we’re seeing a slow paradigm shift from larger testing events that happen at a single point in time to smaller, online testing events happening periodically. What do you think?

5 Things I Learned at the European Association of Test Publishers Conference Last Week

Posted by John Kleeman

I just attended the Association of Test Publisher’s European conference (EATP), held last week in Madrid, and wanted to share some of what I learned.

The Association of Test Publishers (ATP) is the trade association for the assessment industry and promotes good practice in assessment. Questionmark have been members for a long time and I am currently on their board of directors. The theme of the conference was “Transforming Assessments: Challenge. Collaborate. Inspire.”

Panel at European Association of Test Publishers

As well as seeing a bit of Madrid (I particularly enjoyed the beautiful Retiro Park), here are some things I learned at the conference. (These are all my personal opinions, not endorsed by Questionmark or the ATP).

1. Skills change. One area of discussion was skills change. Assessments are often used to measure skills, so as skills change, assessments change too. There were at least three strands of opinion. One is that workplace skills are changing rapidly – half of what you learn today will be out of date in five years, less if you work in technology. Another is that many important skills do not change at all – we need to collaborate with others, analyze information and show emotional resilience; these and other important skills were needed 50 years ago and will still be needed in 50 years’ time. And a third suggested by keynote speaker Lewis Garrad is that change is not new. Ever since the industrial revolution, there has been rapid change, and it’s still the case now. All of these are probably a little true!

2. Artificial Intelligence (AI). Many sessions at the conference covered AI. Of course, a lot of what gets called AI is in fact just clever marketing of smart computer algorithms. But nevertheless, machine learning and other things which might genuinely be AI are definitely on the rise and will be a useful tool to make assessments better. The industry needs to be open and transparent in the use of AI. And in particular, any use of AI to score people or identify anomalies that could indicate test cheating needs to be very well built to defend against the potential of bias.

3. Debate is a good way to learn. There were several debates at the conference, where experts debated issues such as performance testing, how to detect fraud and test privacy vs security, with the audience voting before and after. As the Ancient Greeks knew, this is a good format for learning, as you get to see the arguments on both sides presented with passion. I’d encourage others to use debates for learning.

4. Privacy and test security genuinely need balance. I participated in the privacy vs test security debate, and it’s clear that there is a genuine challenge balancing the privacy rights of individual test-takers and the needs of testing organizations to ensure results are valid and have integrity. There is no single right answer. Test-taker rights are not unlimited. And testing organizations cannot do absolutely anything they want to ensure security. The growing rise of privacy laws including the GDPR has brought discussion about this to the forefront as everyone seeks to give test-takers their mandated privacy rights whilst still being able to process data as needed to ensure test results have integrity. A way forward seems to be emerging where test-takers have privacy and yet testing organizations can assert legitimate interests to resist cheating.

5. Tests have to be useful as well as valid, reliable and fair. One of the highlights of the conference was a CEO panel, where Marten Roorda, CEO of ACT, Norihisa Wada, a senior executive at EduLab in Japan, Sangeet Chowfla, CEO of the Graduate Management Admission Council and Saul Nassé, CEO of Cambridge Assessment gave their views on how assessment was changing. I moderated this panel (see picture below) and it was great to hear these very smart thought leaders talk of the future.  There is widespread agreement that validity, reliability and fairness are key tenets for assessments , but also a reminder that we also need “efficacy” – i.e. that tests need to be useful for their purpose and valuable to those who use them.

There was a huge amount of other conference conversations including sessions on online proctoring, test translation, the update to the ISO 10667 standard, producing new guidelines on technology based assessment and much, much more.

I found it challenging, collaborative and inspiring and I hope this blog gives you a small flavor of the conference.

Ten Key Considerations for Defensibility and Legal Certainty for Tests and Exams

John KleemanPosted by John Kleeman

In my previous post, Defensibility and Legal Certainty for Tests and Exams, I described the concepts of Defensibility and Legal Certainty for tests and exams. Making a test or exam defensible means ensuring that it can withstand legal challenge. Legal certainty relates to whether laws and regulations are clear and precise and people can understand how to conduct themselves in accordance with them. Lack of legal certainty can provide grounds to challenge test and exam results.

Questionmark has just published a new best practice guide on Defensibility and Legal Certainty for Tests and Exams. This blog post describes ten key considerations when creating tests and exams that are defensible and encourage legal certainty.

1. Documentation

Without documentation, it will be very hard to defend your assessment in court, as you will have to rely on people’s recollections. It is important to keep records of the development of your tests and ensure that these records are updated so that they accurately reflect what you are doing within your testing programme. Such records will be powerful evidence in the event of any dispute.

2. Consistent procedures

Testing is more a process than a project. Tests are typically created and then updated over time. It’s important that procedures are consistent over time. For example, a question added into the test after its initial development should go through similar procedures as those for a question when the test was first developed. If you adopt an ad hoc approach to test design and delivery, you are exposing yourself to an increased risk of successful legal challenge.

3. Validity

Validity, reliability and fairness are the three generally accepted principles of good test design. Broadly speaking, validity is how well the assessment matches its purpose. If your tests and exams lack validity, they will be open to legal challenge.

4. Reliability

Reliability is a measure of precision and consistency in an assessment and is also critical.There are many posts explaining reliability and validity on this blog, one useful one is Understanding Assessment Validity and Reliability.

5.  Fairness (or equity)

Probably the biggest cause of legal disputes over assessments is whether they are fair or not. The International standard ISO 10667-1:2011 defines equity as the “principle that every assessment participant should be assessed using procedures that are fair and, as far as
possible, free from subjectivity that would make assessment results less accurate”. A significant part of fairness/equity is that a test should not advantage or disadvantage individuals because of characteristics irrelevant to the competence or skill being measured.

6. Job and task analysis

The type of skills and competence needed for a job change over time. Job and task analysis are techniques used to analyse a job and identify the key tasks performed and the skills and competences needed. If you use a test for a job without having some kind of analysis of job skills, it will be hard to prove and defend that the test is actually appropriate to measure someone’s competence and skills for that job.

7. Set the cut or pass score fairly

It is important that you have evidence to reasonably justify that the cut score used to divide pass from fail does genuinely distinguish the minimally competent from those who are not competent. You should not just choose a score of 60%, 70% or 80% arbitrarily, but instead you should work out the cut score based on the difficulty of questions and what you are measuring.

8. Test more than just knowledge recall

Most real-world jobs and skills need more than just knowing facts. Questions which test remember/recall skills are easy to write but they only measure knowledge. For most tests, it is important that a wider range of skills are included in the test. This can be done with conventional questions that test above knowledge or with other kinds of tests such as observational assessments.

9. Consider more than just multiple choice questions

Multiple choice tests can assess well; however in some regions, multiple choice questions sometimes get a “bad press”. As you design your test, you may want to consider including enhanced stimulus and a variety of question types (e.g. matching, fill-in-blanks, etc.) to reduce the possibility of error in measurement and enhance stakeholder satisfaction.

10. Robust and secure test delivery process

A critical part of the chain of evidence is to be able to show that the test delivery process is robust, that the scores are based on answers genuinely given by the test-taker and that there has been no tampering or mistakes. This requires that the software used to deliver the test is reliable and dependably records evidence including the answers entered by the test-taker and how the score is calculated. It also means that there is good security so that you have evidence that the right person took the test and that risks to the integrity of the test have been mitigated.

For more on these considerations, please check out our best practice guide on Defensibility and Legal Certainty for Tests and Exams, which also contains some legal cases to illustrate the points. You can download the guide HERE – it is free with registration.

Defensibility and Legal Certainty for Tests and Exams

John KleemanPosted by John Kleeman

Questionmark has just published a new best practice guide on Defensibility and Legal Certainty for Tests and Exams. Download the guide HERE.

We are all familiar with the concept of a chain of custody for evidence in a criminal case. If the prosecution seeks to provide evidence to a court of an object found at a crime scene, they will carefully document its provenance and what has happened to it over time, to show that the object offered as evidence at court is the object recovered from the crime scene.

There is a useful analogy between this concept and defensibility and legal certainty in tests and exams. Assessments have a “purpose” or a “goal”, for example, the need to check a person’s competence before allowing them to perform a job task. It is important that an assessment programme defines its purpose clearly, ensures that this purpose is then enshrined in the design of the test or exam, and checks that the assessment and delivery is consistent with the defined purpose. Essentially, there should be a chain from the purpose to design to delivery to decision, which makes the end decision defensible. If you follow that chain, your assessments may be defensible and legally certain; if that chain has breaks or gaps, then your assessments are likely to become less certain and more legally vulnerable.

Defensibility of assessments

Defensibility, in the context of assessments, concerns the ability of a testing organisation to withstand legal challenges. These legal challenges may come from individuals or groups who claim that the organisation itself, the processes followed (e.g., administration, scoring, setting pass scores, etc.), or the outcomes of the testing (e.g., a person is certified or not) are not legally valid. Essentially, defensibility has to do with the question: “Are the assessment results, and more generally the testing program, defensible in a court of law?”.

Ensuring that assessments are defensible means ensuring that assessments are valid, reliable and fair and that you have evidence and documentation available to demonstrate the above, in case of a challenge.

Legal certainty for assessments

Legal certainty (“Rechtssicherheit” in German) means that the law (or other rules) must be certain, in that the law is clear and precise, and its legal implications foreseeable. If there is legal certainty, people should understand how to conduct themselves in accordance with the law. This contrasts with legal indeterminacy, where the law is unclear and may require a court’s ruling to determine what it means

  • Lack of legal certainty can provide grounds to challenge assessment results. For instance many organisations have rules for how they administer assessments or make decisions based on the results of assessments. A test-taker might claim that the organisation has not followed its own rules or that the rules are ambiguous.
  • Some public bodies are constrained by law in which case they can only deliver assessments in a way that laws and regulations permit, and if they veer from this, they can be challenged under legal certainty.
  • Legal certainty issues can also arise if the exam process goes awry. For example, someone might claim that their answers have been swapped with those of another test-taker or that the exam was unfair because the user interface was confusing, e.g. they unintentionally pressed to submit their answers and finish the exam before actually intending to do so.

The best practice guide describes the principles and key steps to make assessments that are defensible and that provide legal certainty, and which are less likely to be successfully challenged in courts. The guide focuses primarily on assessments used in the workplace and in certification. It focuses particularly on legal cases and issues in Europe but will also be relevant in other regions.

You can download the guide HERE – it is free with registration.