Get trustable results: How many test or exam retakes should you allow?

John Kleeman HeadshotPosted by John Kleeman

How many times is it fair and proper for a participant to retake an assessment if they fail?

One of our customers asked me about this recently in regard to a certification exam. I did some research and thought I’d share it  here.

For a few kinds of assessments, you would normally only allow a single attempt, typically if you are measuring something at a specific point in time. A pre-course or post-course test might only be useful if it is taken right before or right after a training course.

Regarding assessments that just give retrieval practice or reinforce learning, you needn’t be concerned. It may be fine to allow as many retakes as people want. The more times they practice answering the questions, the more they will retain the learning.

But how can you decide how many attempts to allow at a certification assessment measuring competence and mastery?

Consider test security

Retakes can jeopardize test security. Someone might take and retake a test to harvest the items to share with others. The more retakes allowed, the more this risk increases.

International Test Commission draft security guidelines say:

“Retake policies should be developed to reduce the opportunities for item harvesting and other forms of test fraud. For example, a test taker should not be allowed to retake a test that he or she “passed” or retake a test until a set amount of time has passed.”

Consider measurement error

All assessment scores have measurement error. A certification exam classifies people as having mastery (pass) or not (failing), but it doesn’t do so perfectly.

If you allow repeat retakes, you increase the risk of classifying someone as a master who is not competent, but  you also decrease the risk of classifying a competent person as having failed. This is because someone can suffer test anxiety or be ill or make a stupid mistake and fail the test despite being competent.

Require participants to wait for retakes

It’s usual to require a time period to elapse before a retake. This  stops people from  using quick, repeated retakes to take unfair advantage of measurement error. It also encourage reflection and re-learning before the next attempt. Standard 13.6 in the Standards for Educational and Psychological Testing says:

“students. . . should have a reasonable number of opportunities to succeed. . . the time intervals between the opportunities should allow for students to have the opportunity to obtain the relevant instructional experiences.”

If we had a perfectly reliable assessment, there would be no concern about multiple attempts. Picking the number of attempts is a compromise between what is fair to the participants and the limitations of our resources as assessment developers.

Think about test preparation

Could your retake policy affect how people prepare for the exam?

If retakes are easily available, some participants might prepare less effectively, hoping that they can “wing it” since  they can retake at will.  On the other hand, if retakes are limited, this could increase test anxiety and stress. It could also increase the motivation to cheat.

What about fairness?

Some people suffer test anxiety, some people make silly mistakes on the test or use poor time management, and some may be not at their full capacity on the day of the exam. It’s usually fair to offer a retake in such situations. If you do not offer sufficient opportunities to retake, this will impact the face validity of the assessment: people might not consider it fair.

If your exam is open the public, you may not be able to limit retakes. Imagine a country where you were not allowed to retake your driving test once you’d failed it 3 times! It might make the roads safer, but most people wouldn’t see it as equitable.

In my next post on this subject, I will share what some organizations do in practice and offer some steps for arriving at an answer that will be suitable for your organization.

Ten tips on reducing test anxiety for online test-takers

Picture of lady biting her nailsJohn Kleeman HeadshotPosted by John Kleeman

I’ve been reading about test anxiety (concern or worry by participants sufficiently severe that it impacts test performance). I’d like to share some tips on how online assessment sponsors can reduce its likelihood.

When creating and delivering tests, you seek to accurately measure knowledge, skills or abilities. Often you want to check competence or mastery for workplace or compliance reasons. If some of your participants have severe test anxiety, this doesn’t just disrupt them, it makes your test less accurate in measuring real performance. You might end up failing someone who is competent, just because anxiety affects their test performance.

Many studies (for example here) report that online tests cause less test anxiety than paper ones. Here are some suggestions on reducing test anxiety:

1. Some people have anxiety about a test because they haven’t mastered the subject being tested. Provide a clear description of what each test covers before the time of the test, and provide study resources or instruction to allow people to master the subject.

2. Test anxiety can also feed on unknowns, for instance on unfamiliarity with the test or believing untrue myths. Share information about the test’s purpose and what you do to make it fair. Also share information about the content: how many questions, how the scoring works, how much time is available and so on. Explain what happens if someone fails – for instance is it possible to retake?

3. It’s hugely valuable to provide practice tests that participants can try out before the real test. This will tell them where they are strong and weak and allow them to gain confidence in a less stressful environment prior to the real test. See my article 10 reasons why practice tests help make perfect exams for other reasons why practice tests are useful.

4. Give participants an opportunity to practice using the same type of computer, mouse, keyboard and user interface as will be used for the real test. This familiarizes them with the test environment and reduces potential anxiety, particularly for those who are less computer literate. If you are using Questionmark to deliver the test, make practice sessions available with the same template settings and the same types of questions. (Sometimes this is done with a fun quiz on a different topic, just to get people accustomed to the user interface.)

5. If you provide guidance to test-takers, point to self-help resources for people who have test anxiety. ETS provide a good resource here for instance. Another resource from the University of California is here.

6. Some self-help resources suggest breathing exercises or other exercises people can follow to reduce tension for people who are anxious about tests. Provide an environment where this is practical and train your test administrators and proctors about the prevalence of test anxiety.

7. If you have a way of encouraging test takers to sleep, take exercise and eat healthily, all these things aid a rational approach to taking a test and reducing anxiety.

8. If it works in your programme, consider whether it’s worth having a series of tests rather than a single test, so there is not a single “make or break” moment for participants. A series of tests can have other benefits too. It makes cheating harder, and by spreading out learning and revision, it can make participants retain the learning better.

9. People with disabilities are more likely to suffer test anxiety. Ensure that your program of accommodations takes this into account. See this helpful article on reducing test anxiety for people with disabilities.

10. Above all, create good quality, fair tests. If you follow good practice in authoring your questions and assessments, then there is less to be anxious about, as the test will be a good measure of performance. See Questionmark’s white paper “Five Steps to Better Tests” for some helpful advice in creating tests.

Many Questionmark users provide very effective practice quizzes and tests which help reduce test anxiety, and I hope these tips are helpful, too.

I’d love to hear additional input or suggestions.