Conference Close-up: Assessments That Measure Knowledge, Skill & Ability

Posted by Joan Phaup

I’ve been having a great time talking to presenters at the Questionmark 2010 Users Conference – customers, our keynote speaker and Questionmark staff. I wanted to find out from Howard Eisenberg about the Best Practices presentation he will deliver at the conference on Effectively Measuring Knowledge, Skill and Ability with Well-crafted Assessments

Q: Could you explain your role at Questionmark?

A: I manage Training and Consulting, so I work with our customers to get the most of their assessments and their use of Questionmark Perception. For some that might mean training on how to use the software effectively. For others it might mean providing solutions that allow them to use the software within the context of their current business processes, such as synchronizing the data between the organization’s central user directory and Perception. In some cases we might need to create reports to supplement those that come with the product or do some other custom development. Sometimes we go on site, install the Perception software and set it up within the customers’ LMS and do any troubleshooting right on the spot. Whatever we do, our goal is to ensure customers’ speed to success, getting them operational faster.

Q: What will you be talking about during your Best Practice session in Miami?

A:  Over the years I’ve given presentations on Creating Assessments That Get Results, where I cover the dos and don’ts of writing test items. A question that always comes up during those talks is how to write test content that goes beyond testing information recall…content that tests a person’s ability to perform a task. There are limitations to using software like Perception to do that: certain things simply require that a person perform a task and have someone observe them, so that all the scoring and evaluation is done by an observer or rater. But there are a lot of possibilities for creating computer-scored items that can measure skill and ability rather than just recall of information. This session is designed to give people tools to take their tests to that level. First we need a framework for categorizing knowledge, skills and abilities: what makes a skill a skill and an ability an ability. We’ll help people classify their learning objectives along those lines and look at specific types of questions that can be used to measure skill and ability. The questions that provide this kind of measurement expand upon the question types that are supported in Questionmark Perception—selected response types as well as constructed responses.   We’ll use several real-world examples to illustrate how questions of this nature go beyond recall of knowledge and go to skill and ability.

Q: What are you looking forward to at the conference?

A: I am really looking forward to meeting customers and in some cases reconnecting with customers I’ve gotten to know over the years. That’s really a highlight for me…reconnecting with our great customers. I am consistently amazed and impressed about how passionate our customers are about what they do with our software and how smart they are in using it. Every year, after talking with a customer or sitting in on a case study, I come away thinking, Wow! That was really clever! So I’m looking forward to hearing those kinds of stories again this year.

The conference program is nearly finalized and includes case studies, tech training and best practice sessions for every experience level.  Check it out and plan to join us March 14 – 17 in Miami!

Leave a Reply