With the program for the Questionmark European Users Conference filling up quickly, I got on the phone this week with one of the case study presenters, Onno Tomson.
Onno is a senior advisor at ECABO (Centre of Expertise on Vocational Education, Training and Labour Market for the economic/administrative, ICT and security professions.). Based in The Netherlands, ECABO runs a large number of tests and certifications for European organizations – everything from safety certifications for workers in the oil industry to required tests for financial advisors.
Onno told me a little about ECABO’s longtime use of Questionmark and some of the new things they are doing. His presentation, Questionmark Can Be More than Questionmark, will explain how ECABO connects Questionmark Perception with third-party applications. It will also offer advice about how using the right tools, options and connections can improve efficiency.
Here’s our conversation in a nutshell:
Could you tell me how ECABO uses Questionmark Perception?
We create and deliver exams for four big organizations with participants in different countries. We are using Perception to create all the exams, so Questionmark is at the heart of our exam department. It’s our main software for creating all the content for several large item banks. Without Questionmark we would be doing this all by paper and pencil! Right now, we are helping one organization transition from paper-based tests to computerized tests. We use Questionmark for them too, using what we call “Questionmark to Paper “(Q2P). This is a custom-built application that uses Application Protocol Interfaces (APIs) to produce Questionmark Perception tests in PDF format and scan the results into Perception for reporting and analysis.
How many tests do you deliver each year, and in how many different languages?
We deliver tests in 12 different languages and still growing. We run 150,000 assessments a year just in the one organization running paper-based tests and transitioning to CBE (computer-based exams). In addition, we’re running 5,000 online assessments and in the end we expect to be running at least 100,000 online assessments per year. Paper-based assessments will always remain but the amount will decrease.
How do you optimize the efficiency and quality of your assessments?
We try to automate as much as possible all the steps in the process. For example, people register online and from that moment automatically a new group of participants is created and linked to a specific test center. And administrators of that test center automatically get direct access to new groups they are responsible for. The participants are also automatically scheduled for this assessment. QMWISe (Questionmark Web Integration Services environment) helps us make API connections with third-party applications, and we make the most of our Questionmark investment by using Questionmark Live, Translation Manager, Printing and Scanning, Result Warehousing and other options, plus some custom-built tools. These all help us with construction, distribution and reporting so that we can meet customer demands cost-effectively.
Could you tell me a little more about connections with third-party applications?
One of the main third-party apps is the app we created for producing paper-based exams. This application communicates using Questionmark APIs, and it can handle all kinds of variables and lay-outs, bubble sheets included. We also have a registration system that needs to talk to Questionmark, another app for sending records to our back office for invoicing, and one for connecting with a national database for certification candidates. All of these apps need to communicate with Questionmark, and we have learned a lot about using QMWISe to achieve this. I’ll be talking about that during my presentation.
How are you using Questionmark Live?
Questionmark Live is our major platform for our subject matter experts. As far as possible, construction of questions is going into Questionmark Live and from there into our production database. We have between 100 and 200 SMEs. So being able to use Questionmark Live means that we are now getting more questions in the proper format for use in Perception instead of having to convert from documents in other formats. It’s a lot easier for us. And our SMEs are learning to use Questionmark Live for collaboration.
What are you looking forward to at the conference?
Meeting with the Questionmark staff, of course! And one of the main things I’m doing at the conference is going to Product Central and discussing how to equip Questionmark for the future. One of the things we would like to discuss is analyzing assessments using Item Response Theory.
The conference will take place in Brussels from the 9th to the 11th of October.
Check out the conference schedule and register soon.