Posted by Chloe Mendonca
It’s always interesting to talk with customers and discover how they use assessments to meet their business needs. My recent conversation with Onno Tomson, – Senior Advisor at ECABO (soon to be known as eX:plain), gave me some background into the world of metadata and its effectiveness in the prevention of cheating.
Onno will present a session at the Questionmark 2013 Users Conference in Barcelona from 10-12 November. I was interested to know more about his presentation, “Advanced Item Banking and Psychometric Analyses with Randomly Generated Assessments.”
Could you tell me about your organization and how you currently use Questionmark assessments?
eX:plain is a knowledge centre for vocational education and professional development. We develop and hold examinations and supply the appropriate content in training for a variety of professions. Questionmark is one of our main applications. We use it for the construction and delivery of our exams, both online and on paper. We average about 200, 000 exams per year.
Why do you use randomly generated questions in your assessments, and could you tell me how this works?
The randomization of questions in our exams is an essential way to prevent cheating. We have a large number of questions included in each bank which means that each individual taking the test will receive a completely different set of questions making it impossible for them to copy one another’s answers.
What problems can arise when drawing questions from large item banks to create your exams?
First, there is the challenge of keeping a good overview of the content.
Second we need to work with large amounts of metadata to make sure the right question pops up at the right spot in the right exam and for management and maintenance purposes.
Selecting questions on multiple metadata values can have the risk of negatively impacting performance. At the conference I will share my experiences and solutions to these important issues.
What do you hope people will take away from your breakout session?
A broader insight into how Questionmark deals with metadata, how it affects your grip on content and how it can affect performance.
What do you hope to take away from the conference?
I’m looking forward to getting some updates on the development of Questionmark and my network and helping to influence the future product features by speaking with the Questionmark management team.