Posted by John Kleeman
Something I enjoy most about our users conferences is the chance to learn from experts about good practice in assessments. Most of our customers have deep knowledge and insightful practical experience, so there is always much to learn.
Here are some tips I picked up last week at our recent European Users Conference in Barcelona.
1. Make sure to blueprint. It’s critical to have a detailed design (often called a blueprint) for an assessment – or as one user shared, “Without a blueprint, you don’t have an assessment”.
2. Network to get SMEs. With technology changing quickly, if your assessments assess IT or other new technology, the content changes very quickly and the quality of your subject matter experts (SMEs) who create and review items is critical. As an assessment owner, use networking skills to get the right SMEs on board; getting them engaged and building trust are essential.
3. Test above knowledge. Develop questions that test application or comprehension, for instance using scenarios. They are more likely to make your test valid than questions that simply test facts.
4. Give employees ownership of their own compliance testing. If employees have to take annual refresher tests, give them the responsibility to do so and encourage sel- learning and pre-reading. Give them plenty of time (e.g. 6 weeks’ warning), but make it their responsibility to take and pass the test in the window, not yours to keep on reminding them.
5. Gather feedback from participants. Make sure you solicit feedback from your participants on tests and the testing experience. That way you will learn about weak questions and how to improve your testing process. And you also make participants feel that the process is fairer.
6. Use job/task analysis. Asking questions about jobs and tasks is the best way to specify the criteria used to judge competency or proficiency. These questions can be automated in Questionmark right now. Watch this space for improvements coming to make this easier.
7. Look at Questionmark Live for item review workshops. If you have any informal or informal process for having groups of people working on or reviewing items, look at Questionmark Live. It’s free to use, has great group working capability and improves productivity. A lot of organizations are having success with it.
9. Look for live data, not just your rear view mirror. Data is important – without measurement we cannot improve. But make sure the data you are looking at is not dead data. Looking into the rear view mirror of what happened in the past doesn’t help as much as using reports and analytics from Questionmark to discover what is happening now, and use that data to improve things.
I hope some of these tips can help you in your work with assessments.
I will write in the spring about the tips I gather at the 2014 U.S. Users Conference in San Antonio!