Evidence that assessments improve learning outcomes
Posted by John Kleeman
I’ve written about this research before, but it’s a very compelling example and I think it’s useful as evidence that giving low stakes quizzes during a course correlates strongly with improved learning outcomes.
The study was conducted by two economics lecturers, Dr Simon Angus and Judith Watson, and is titled Does regular online testing enhance student learning in the numerical sciences? Robust evidence from a large data set. It was published in the British Journal of Educational Technology Vol 40 No 2, 255-272 in 2009.
Angus and Watson introduced a series of 4 online, formative quizzes into a business mathematics course, and wanted to determine whether students who took the quizzes learned more and did better on the final exam than those who didn’t. The interesting thing about the study is that they used a statistical technique which allowed them to estimate the effect of several different factors, and isolate the effects of taking the quizzes from the previous mathematical experience of the students, their gender and their general level of effort to determine which impacted the final exam score most.
You can see a summary of their findings in the graph below, which shows the estimated coefficients for four of the main factors, all of which had a statistical significance of p < 0.01.
You can see from this graph that the biggest factor associated with final exam success was how well students had done in the midterm exam, i.e. how well they were doing in the course generally. But students who took the 4 online quizzes learned from them and did significantly better. The impact of taking or not taking the quizzes was broadly the same as the impact of their prior maths education: i.e. quite reasonable and significant.
We know intuitively that formative quizzes help learning, but it’s nice to see a statistical proof that – to quote the authors – “exposure to a regular (low mark) online quiz instrument has a significant and positive effect on student learning as measured by an end of semester examination”.
Another good resource on the benefits of assessments to check out is the white paper, The Learning Benefits of Questions. In it, Dr. Will Thalheimer of Work-Learning Research reveals research that shows that questions can produce significant learning and performance benefits, potentially improving learning by 150% or more. The white paper is complimentary after registration.
John Kleeman will discuss benefits and good practice in assessments at the 2015 Users Conference in Napa Valley, March 10-13. Register before Dec. 17 and save $200.