10 reasons why practice tests help make perfect exams

John Kleeman HeadshotPosted by John Kleeman

Giving the opportunity for candidates / participants to take a practice or mock version of an exam before they take the real thing has huge benefits for all stakeholders. Here are 10 reasons why including practice tests within your exam programme will improve it.

1. Most importantly, practice tests tell candidates which topics they have not mastered and encourage them to focus future learning on weak areas.

2. Almost as important, practice tests tell candidates which topics they have already mastered. They can then direct their learning to other areas and spend minimal further time on the topics they already know.

3. Practice tests can also feed back to the instructional team the strengths and weaknesses of each candidate and for the candidate group. It can tell which topics have been successfully learned and which areas need more work.

image4. It’s well understood in psychology that you are more likely to retain something if you learn it spaced (separated) over time. Since practice tests stimulate revision and studying, they encourage earlier learning and so space out learning, which is likely to improve retention. See this Slideshare for more information on how assessments can help space out learning.

5. The accuracy and fairness of exams can be impacted by some candidate’s fear or anxiety around the exam process. Practice tests can reduce test anxiety. To quote ETS on test anxiety:

“The more you are accustomed to sitting for a period of time, answering test questions, and pacing yourself, the more comfortable you will feel when you actually sit down to take the test.”auth-collab-350x200

6. Accuracy and fairness can also be impacted by problems with familiarization or incompatibilities with the computers and software used for the testing. If the same equipment and software can be used in practice, this greatly reduces the chance of problems.

7. Taking a test doesn’t just measure how much you know, it helps reinforce the learning and make it more likely that you can retrieve the same information later. It’s a surprising fact that taking a test can actually be more beneficial to learning than spending the same amount of time studying. See Evidence from Medical Education that Quizzes Do Slow the Forgetting Curve for one of many research studies showing this.

8. Giving formative or practice tests seems to improve learning as well as final exam results. See Evidence that topic feedback correlates with improved learning or  Where’s the evidence for assessment? for a couple of articles with evidence of this.

9. Such tests are consistent with good practice and with assessment standards. For example the international standard on delivering assessments in the workplace ISO 10667 states:

“The service provider shall … where appropriate, provide guidance on ways in which the assessment participant might prepare for the assessment, including access to approved or recommended sample and practice materials”

10. It is crucial that exams  are fair and that they are seen to be fair. By providing practice tests, you remove the mystique from your exams and allow people to see the question styles, to practice the time planning required and to have a fair view of what the exam consists of. It helps level the playing field and promotes the concept of a fair exam, open to and equal for all.

Not all these reasons apply in every organization, but most do.  I hope this article helps remind you why practice tests are valuable and encourages their use.

Don’t believe quizzes and tests help retention? Heed the evidence

Posted by John Kleeman

I’ve been sharing the new evidence from cognitive psychology at several conferences in the last year. This peer-reviewed, scientific evidence shows that taking a quiz or test enhances learning and slows the forgetting curve. Although many people are enthused and fascinated, some are doubtful. And a few turn away because they regard cognitive psychology as being out of fashion.

Of course, it takes time for all new evidence to get accepted, and I’d like to share some of the questions comments and objections I’ve been hearing lately.

In case you’re not familiar with the evidence, here are the slides from my presentation at Online Educa in Berlin in December (or you can also see them on SlideShare), or check out related blog articles: here, here and here).

ONLINE EDUCA 2011

View more presentations from Questionmark

You could also come to my session at the Questionmark Users Conference in New Orleans in March 2012, where I’ll be talking about the benefits of spacing out learning and how assessments can help here.

So what are the objections I’ve been hearing recently? And how do I respond?

“We test our students too much already, we don’t need more tests”

Some people confuse the benefits of quizzes and tests as retrieval practice with the over-use of standardized tests in schools. They are two different things – and it can be helpful to use a word other than “tests” for retrieval practice, for instance quizzes or exercises.

“We don’t have time for more tests and quizzes”

A complaint can be that the course is too full to find space for more quizzes. But since the evidence shows that spending time taking a quiz helps learning more than re-studying a subject, it may be worth adjusting things. If you want your learners to retain what you teach, not just learn and forget it, you need to give quizzes or other retrieval practice

“We teach creativity or other higher level skills. not facts”

There are two answers to this. One is that the research implies that quizzes and tests encourage retention of concepts as well as facts. The other is that all disciplines need knowledge of basic facts and vocabulary to build creativity on top.

“How valid is the research?”

Some people would like to see more studies with larger numbers of participants and in the real world. Or they wonder whether increased forgetting of the material not tested counter-balances the positive effects of quizzes as retrieval practice. These are good questions; this research has mostly happened in the last few years, and I would like to see more of it. But the basic concepts go back hundreds of years. The evidence for quizzes directly helping learning, particularly if they include feedback, is strong.

“Our learners don’t like being tested”

Unfortunately the retrieval practice benefit of quizzes helps more in the long term than the short term. So there is a paradox that if you take a quiz or test you might feel it’s not helping when it is. This can be a challenge. Part of the answer is to make quizzes enjoyable and part is to educate learners on the benefits of retrieval practice.

“Our employees have privacy concerns about being tested”

There are ways to deal with this, for example by following a respected standard like ISO 10667. If privacy is a concern, you can make retrieval practice quizzes anonymous. This still gives learning benefit but makes it harder to track.

“Psychologists don’t know anything about pedagogy”

This may be true, but if psychologists can demonstrate (as they seem to in peer-reviewed studies) that learners retain information better in memory by learning in one way or another, then sensible instructors will learn from the research and apply it to their pedagogy.

“Cognitive psychology is not fashionable in education”

This is the one objection I don’t have a good answer to! I may be missing some of the history between the two disciplines, but it seems to me that we should base instruction and learning on what the evidence says and not on fashion or opinion. But if someone believes otherwise I’m not sure how to persuade them. Any suggestions?

How Questionmark can help you comply with the new ISO 10667 standard

Posted by John Kleeman

In the last month or two, ISO, the International Standards Organization has introduced an important new standard covering assessments in the workplace, ISO 10667.

ISO 10667’s full title is “Assessment service delivery — Procedures and methods to assess people in work and organizational settings”. It’s a wide-ranging standard that covers all kinds of assessments in the workplace from appraisals and coaching through psychological tests, it includes knowledge and skill quizzes, tests and exams in recruitment, training and compliance.

There are three key roles defined in the standard: Service provider, Client and Assessment participant as shown in the diagram below.

Three key roles defined in ISO 10667

The standard is split into two parts. Part 1 sets requirements for the Client and part 2 sets requirements for the Service provider. The standard is unfortunately not available for free download, it has to be purchased from a standards organization, here are some potential places to obtain it:

The standard does not apply in education, but could be useful for corporate and government users of assessment. Some advantages of being compliant with ISO 10667 are:

  • Assessment in the workplace lets you see into the minds and hearts of employees and other stakeholders, but also potentially discloses sensitive and private data that requires privacy. The standard covers issues such as informed consent and what to do to ensure proper treatment of assessment data; it will help you ensure you are following best practice.
  • If you make decisions on employees (e.g. hire, fire, promote) as a result of assessments, then following the standard could be helpful if any such decisions are disputed (by trade unions, courts of law or individual employees).
  • When people cheat at assessments, they often rationalize the cheating to themselves that it’s fair to do so because the assessment programme is unfair. Publicly committing to a standard like ISO 10667 can help you promote your assessment programme as open, fair and reasonable and so make it less likely that people will rationalize cheating on it.
  • The standard has been written by experts in assessment, following it lets you learn and follow best practice.

To help people considering using Questionmark to comply with ISO 10667, we have produced a short white paper on the standard and how Questionmark can help you meet it. The white paper is free with registration and is available here.

Influence the new ISO 10667 standard on workplace assessment

john_smallPosted by John Kleeman

An important new ISO international standard for assessments in the workplace is in its final stages and a draft is now available for consultation.

The standard is called ISO 10667 and covers assessment service delivery: procedures and methods to assess people in work and organizational settings. There are two parts, one for service providers and one for clients: organizations that use assessments. It has very broad scope, from assessment in appraisals and coaching through 360s and psychological assessments as well as compliance and training assessments, and it covers both paper and computer-delivered assessments.

image

There are two closely related parts of the standard and it sets out good practice and guidelines for areas such as the following:

  • Agreeing procedures between different stakeholders in the assessment process
  • Planning assessment delivery formally
  • Getting informed consent from assessment participants
  • Privacy and data protection on assessment results
  • Security and confidentiality
  • Ensuring that reports arising from the assessment are based fairly on what the assessment measures
  • Providing appropriate assessment feedback
  • Guidelines on rights and responsibilities of assessment participants

ISO 10667 is the result of many years’ work by an international committee with representatives from many countries including the US and several in Europe. I have been involved in a very small way with the committee and have been impressed by the professionalism and knowledge of those responsible for writing the standard.

When the standard is formally published, ISO 10667 will provide an opportunity for organizations to put in place a consistent quality standard for their use of assessment, which if we can get it right could help greatly in assessment consistency and fairness. For those involved with assessment in the workplace, there is also likely to be pressure from stakeholders to follow the standard, so reviewing it in advance could be sensible.

ISO is organized via national committees, and there is no single international place to get hold of the draft and provide comments. You have to do this within your own country. If you are in the UK, then you link via the British Standards Institution (BSI) to view Part 1  and Part 2  (needs registration) and comment. If you are in the US, the Association of Test Publishers is administering the review of the draft standard, and if you want to comment, you should contact wgharris@testpublishers.org in the first instance. If you are in other countries, you should contact your national standards organization.

The degree to which the final standard can help improve the quality of assessment in work and organizational settings will depend on the standard being reasonable and practical in terms of the demands it makes on all of us, and I’d encourage Questionmark users who are interested in this area to provide input to the consultation process to help ensure ISO gets it right.