Performance testing is to certifications as simulation is to learning

Howard Eisenberg HeadshotPost by Howard Eisenberg

I just attended the Performance Testing Council Summit. Performance testing is “testing by doing.”  Exam developers create performance items that require candidates to actually perform real-world, authentic task not multiple-choice questions that have only one best answer or allow a low-ability candidate to guess the correct answer.  The outcome of the task is then evaluated to determine a score, or how well the candidate performed.

All but one of the attendees of this meeting representing certification programs were from software/IT companies. The IT domain lends itself very well to the adoption of performance testing. Advances in virtualization and software-as-a-service make it possible to provision “testing labs” with specific characteristics and traits in minutes and at low costs.  Moreover, as these labs can be hosted in the cloud nowadays, there’s no need for a candidate to travel to a specific location to take an exam.  This means that IT, performance-based certifications can and indeed ARE being delivered online and on-demand, with the help of remote proctoring and other technology-enabled security controls.

sunWe’d like to hear from you! The call for proposals is officially open for the Questionmark 2016 Users Conference. If you have an experience you would like to share with the Questionmark community, please submit a presentation proposal here: click here.

These labs are the real-world context in which an IT professional works.  It’s not a simulation of the software, tools, network connections, etc.: it is the real thing.  As such, it’s arguably a more valid methodology for assessing an IT professional’s ability to perform the tasks required by the job.  So using a performance exam for an IT certification makes sense.

Alas, not all certification exams and the professional domains they represent are as well suited to performance testing. It’s not as easy to recreate the environment in which a registered nurse performs his or her daily duties, for example.  In other domains where technology is not center-stage, Questionmark’s customers have historically done the next best thing.  What’s that, you ask?  Well, it’s simulating the performance environment within the test. And if high-fidelity simulation is not cost-effective to develop, then it’s using real-world exhibits, artifacts, and scenarios expressed through multimedia to bring as much of the performance context/environment to the test as is feasible and cost-effective.

Performance testing is to certifications as simulation is to learning.  It’s that “holy grail.”  If we can make the exam look and feel like the job, then it will have the greatest potential to be the truest measure of ability.  If we can make the training look and feel like the job, then it will have the greatest potential to adequately prepare the employee.  (I say “potential” only because the instrument or the simulation must still be well-designed).

I know that many Questionmark customers have struggled to attain this ideal. That is the reality of working with budgets, timelines and other limited resources.  But I’m willing to bet that many customers have creatively worked around these challenges to create valid tests and exams that provide solid measurement value to the programs in which they are used.

sunIf you have a story to tell about such challenges and solutions, then please share them with the Questionmark community at the Questionmark 2016 Users Conference. Click here to submit your presentation proposal. *Submission deadline is December 4. Slots are limited.

Leave a Reply