Fair. Valid. Reliable. — Item Writing Guide, Part 1

Doug Peterson HeadshotPosted By Doug Peterson

While many people tend to think of quizzes, tests and exams as being made up of questions, professionals in the assessment industry typically use the term “item” because the question is only a part of the overall interaction with the learner, along with choices, scoring and feedback.

Well-written items can be used to assess what a learner needs to learn prior to a learning event as well as what they *have* learned after the learning event. Well-written items promote learning and memory recall, and help retain knowledge, skills and/or abilities over time. But writing good items isn’t as easy as it looks.

I’ll be devoting a few blog posts to some pointers about item writing and hope you find them helpful.

Today, let’s consider these three important qualities of well-written items – they need to be fair, valid and reliable.

Fair

“Trick” items, or confusing or misleading items do not allow the test-taker to show their true understanding of the subject matter, and stakeholders in the testing process would not be able to trust the results. The goal of an item is not to prove what a test-taker doesn’t know, it’s to allow him or her to show what they *do* know. Also, an item should only test one thing, so you only want to test the participant’s knowledge OR their puzzle-solving abilities, but not both at the same time.

Valid

An item needs to be valid in the context of the assessment. For example, an assessment that tests a learner’s ability to diagnose Alzheimer’s disease should not contain an item about dental hygiene. The dental hygiene item, as well-written as it may be, is simply not valid in that context.

Reliable

Finally, an item needs to be reliable. For example, if you were asking a five-year-old a question that requires the language comprehension skills of a ten-year-old, it’s unlikely that your results would be very reliable. An item must accurately measure the test-taker’s true understanding of the subject matter, repeatedly over time, and reduce the possibility of guessing the correct answer as much as possible.

Please feel free to leave a comment in the area below! In Part 2, we’ll take a look at the different parts of an item as well as some basic guidelines for writing a good stimulus and good choices.

Other posts in this series:

Results Management System Quiz: Test your knowledge!

greg_pope-150x1502

Posted by Greg Pope

Organizations involved in medium and high-stakes testing must employ sound test development, administration and scoring processes to help ensure fair, reliable and valid assessments.

Knowledge Check

But despite everyone’s best efforts, there are times when it’s necessary to review and potentially modify test results to provide information and certificates that fairly reflect what was being measured.That’s where the Questionmark RMS, or Results
Management System comes in: It enables organizations to analyze, edit and publish assessment results in an informed and defensible way.

I have created a quiz on RMS to test your knowledge. Take assessment one and see how well you do. All the answers for the questions are available on the Questionmark web site, so if you study hard you can get a perfect score and impress your friends and colleagues.  Good luck!

.