White Paper: 5 Steps to Better Tests

Joan Phaup HeadshotPosted by Joan Phaup5 white paper

Doug Peterson‘s recent 10-part series about test design and delivery — published here on the blog this winter — offers some great advice about five key stages of testing deployment.

There was so much interest in this subject that we’ve used his posts as the basis for a new white paper designed to help you save time and effort as you produce assessments that truly serve your organization’s needs.

The paper will walk you through these essential steps:

1. Planning the test
2. Creating the test items
3. Creating the test form
4. Delivering the test
5. Evaluating the test

We invite you to download the paper  with our compliments! And do send us any questions you have about it. We’ll be very happy to respond.


Webinar: Creating high-complexity items for performance-based assessments

Joan Phaup HeadshotPosted by Joan Phaup

James Parry

Creating test items that check someone’s ability to apply knowledge — rather just recall what they have learned — presents some special challenges.

James Parry, E-Testing Manager at the U.S. Coast Guard Training Center in Yorktown, Virginia, led a discussion about this at the Questionmark 2013 Users Conference.

I’m pleased to announce that he will be sharing his expertise on this subject during a webinar set for Thursday, June 6 at 1 p.m. Eastern Time.

James will help participants understand the value of high-complexity test items in performance-based tests, explain what makes these questions different from others and offer tips on how to write them well.

Here’s what you can expect to learn about during this free, one-hour webinar:

  • Differences between knowledge testing and performance testing
  • Definitions and comparisons of low-, moderate- and high-complexity test items
  • How performance-based tests relate to Bloom’s Taxonomy and Gagne’s Nine Events of Instruction
  • How to match the complexity of test items to the desired level of cognitive performance
  • Tips for ensuring the quality of high-complexity test items

Sign up today for Creating High-Complexity Test Items.


Writing high-complexity items for performance-based tests

Joan Phaup HeadshotPosted by Joan Phaup

Performance-based tests require complex items that require participants to show that they know how to apply what they have learned.

James Parry from the U.S. Coast Guard will lead a discussion about the challenges involved in writing these types of questions during the Questionmark 2013 Users Conference in Baltimore March 3 – 6: Advanced Question Writing: Creating High-Complexity Test Items.

I asked James for some background on the subject, which is a major interest of his work as Test Development/E-Testing Manager for Coast Guard Training Center Yorktown, Virginia.

Why are higher-complexity test items important?


James Parry

The push in today’s society is towards performance-based testing to ensure a candidate can actually ‘do’ the job and not just recite knowledge about the job.  True performance-based testing involves each individual performing each task while being observed. With our challenging global economy, having both the time and real equipment for each student to perform each task is, most times, cost prohibitive.  Well-written questions that address performance rather than just knowledge can help bridge that gap.

Why is it more difficult to write questions that go beyond checking someone’s knowledge?

Performance-based test items are expensive and time consuming to develop compared to knowledge-based items.  A solution is a hybrid approach which can use a traditional objective-based written test item to check the foundation knowledge at the highest cognitive level possible as a precursor to the actual performance test.  Being able to develop test items at a higher cognitive level is difficult because the designer has to think outside of the traditional multiple choice test item and more towards the performance.

What’s your own experience writing higher-complexity test items?

The Coast Guard requires performance-based testing at all of its entry-level schools to ensure the young men and women going into the field can actually perform the jobs they are trained to do.  As the test development manager I work with the test authors at the schools to help build foundational assessments at various levels of complexity to test the students comprehension as close to the actual performance as possible before actually performing the tasks.

What will be happening during your discussion session?

I will be discussing how both Bloom’s Taxonomy and Gangé’s nine Instructional events relate to both complexity and cognitive levels of testing.  I’ll share an example of how a simple objective can be tested at just about any level of complexity and couple the example with Bloom’s Taxonomy.  Then I’ll ask the participants to develop a test item at all of the complexity and taxonomy levels.

Who do you think will be benefit most from this discussion?

Participants who have a basic understanding of test item development will benefit the most, but novice test developers and training managers will benefit as well.

How can people prepare for this session?

Those who attend Sharon Shrock and Bill Coscarelli’s pre-conference workshop on Criterion-Reference Test Development will have a good foundation to build on – but just reading up on the subject of performance-based testing would be helpful, too. I’d like participants to think of a learning objective in their own area of expertise and how they would like to design items that test more than just rote knowledge.

What do you hope your participants will take away?

Participants should be able to walk away with a better understanding of the challenges associated with solid development of test items that test more than memorization of knowledge.

You’ve attended several Users Conferences. What prompts you to return?

I think this is my 7th or 8th User’s Conference.  Every time I attend I walk away with a better understanding of testing and how Questionmark technologies are being used in creative ways to enhance the testing experience.  The networking opportunities and access to testing experts from around the world are invaluable to me.

Click here to see the complete conference program, and register soon.

Podcast: Options for Organizing Test Items

Posted by Joan Phaup

What’s the most effective way for you to organize test items? Should you stick with topic-based organization? Or would it be more useful to organize by objectives? Lynn Duffield of Verizon Business discusses this dilemma in our latest podcast.