Conference Close-up: John Kleeman on Using Research Findings to Improve Assessments
Posted by Joan Phaup
Current psychological research confirms that answering questions helps people remember what they’ve learned. But how can this research be applied to creating effective quizzes and tests? Questionmark Chairman John Kleeman will address that question during a presentation at the Questionmark Users Conference on Using the Latest Learning Research to Improve Your Questionmark Assessments.
John has taken a keen interest in this research, which he has written about it in a previous post, and an article in yesterday’s New York Times shows that research in this area is getting mainstream attention.
I thought it would do John some good to answer some questions about what he’s been learning and what he plans to share with his audience!
Q: What research findings do you regard as most relevant to the use of assessments in learning?
A: There’s a huge amount of psychological research out there, but what I still find stunning — although the basics have been known for a while — is that if you study something and don’t answer questions about it you are likely to forget it over time, but if you do answer questions it gives you retrieval practice and you are more likely to retain it. So the old-style practice of teaching people something and then quizzing them on it really does work. People who’ve been away from school think, “Oh, I don’t need to take quizzes and tests anymore,” but they really should take quizzes and tests because the process of answering a question and getting retrieval practice really helps you learn. This is an example of a desirable difficulty – something you don’t want to do, but which helps you learn.
Q: Why is retrieval practice so useful?
A: As I understand it, when you answer a question it gives your brain practice in retrieval and makes the pathway from that information to how you can apply it stronger…in layman terms, it makes those brain pathways stronger. The evidence shows that if you do answer things after learning you remember it better. If you want to learn something and retain it, be sure to answer some questions about it. Even if you answer a question only to yourself you are improving your retention.
Q: Is this an academic session?
A: Absolutely not! What I’m looking to do is to highlight some of the research that has been done — including new research from the last few years — and to suggest practical things that people using Questionmark software can to do take advantage of it. I’ll be covering the retrieval effect and also how pre-questions can be effective and on how to use feedback effectively in Questionmark Perception. The session is about how we can practically use the results of research.
Q: Can you tell me about some of the interactive exercises you will be doing during this breakout session?
A: I’ve got a few up my sleeve! One is that I plan to divide the room into two random groups. One part will study something on its own, without answering any follow-up questions; the other will study and answer some questions. A week afterwards I’ll run a voluntary assessment with those two groups and see which group performs better. I want to demonstrate that questions really do help you learn.
Q: What will people get out of the session?
I went to a training course recently. There were about 20 people, and at the end of the course there was an optional test. Only about six people took it. The others probably thought they had better things to do with their time or might have been concerned about their scores! But I want my audience to understand how much value there is in assessment and how it can help people be effective learners. We’re missing something here and there’s a great opportunity for us to do better. I want to empower attendees with insight so that they can go back to their organizations and be more effective.