Randy Bennett (who holds the Frederiksen Chair in Assessment Innovation at ETS – Educational Testing Service) is one of the world’s experts on computerizing assessments. I very much enjoyed his recent keynote at the International Computer Assisted Assessment Conference. With Dr. Bennett’s permission, here is a summary of his presentation.
His key proposal is that we should not use technology in assessment because it is cool or because it is efficient, but to make assessment better. “If we focus on efficiency, we may end up with nothing more than the ability to create existing tests faster, cheaper, and in greater numbers without necessarily making them better.”
Here are his 11 propositions for what technology in assessment should do:
1. Give students more meaningful assessment tasks than are feasible through traditional approaches.
2. Model good instructional practice, including encouraging habits of mind common to proficient performers in the domain.
3. Assess important competencies not measured well in conventional form, e.g. simulations or using a spreadsheet.
4. Measure “problem-solving with technology”, given that the workplace typically requires use of technology.
5. Collect response information that can enlighten substantive interpretation (e.g. the time taken to answer questions).
6. Make assessment fairer for all students including those with disabilities and for non-native language speakers.
7. Explore new approaches to adaptive testing to assess authentically the full range of important competencies not just the middle ranges.
8. Measure more frequently, aggregating information over time to form a summative judgement.
9. Improve the substantive aspects of scoring, for instance use technology to make scoring more effective.
10. Report assessment results in a timely and instructionally actionable manner, including pointing to likely next steps and instructional materials for them.
11. Help teachers and students understand the characteristics of good performance by participating in onscreen marking – for instance mark work and have others review your marks to help you develop understanding.
You can see the full keynote presentation here with several screenshots illustrating what can be done.
I believe that good computerized assessment does much more than simply computerize paper practices, and it’s great to see this thoughtful call to action.