Writing Test Items – Test Design and Delivery Part 4

Posted By Doug Peterson

In Part 1, Part 2, and Part 3 of this blog series, we looked at the planning that goes into developing a test before the first item is ever written. Let’s assume that planning has taken place and it’s time to write some items.

There is a LOT of information out there about writing items. I have a blog on Improving Multiple-Choice Questions and another on Mastering Multiple-Choice Questions, as well as one on True/False questions. Any time anyone asks me about item writing do’s and don’ts, I always mention the book “Criterion-Referenced Test Development” by Shrock & Coscarelli, especially chapter 7. In this blog I’d like to touch on a few item-writing tips that apply to pretty much any item type.

Always strive for clarity and readability. Remember, you only want to test for one thing, so make sure that you’re not testing the learner’s reading ability or comprehension ability as well as the specific piece of knowledge for which you’re testing. Avoid “window dressing” or superfluous information that isn’t necessary to ask the question and get a response. Make sure that you’re truly asking a single question. You don’t want to ask something like “What color is a stop sign and how many sides does it have?” or “Which of the following Dallas attractions is the most popular and why?” You also want to make sure to avoid negative phrasing such as “Which of the following is not an acceptable way to …”, since this increases the cognitive load and introduces confusion without increasing the value of the question.

Use a style guide for consistency. You don’t want anything to distract the learner, so make sure you use the same font size and family, using bolding and italics consistently, etc. Test-takers are nervous enough as it is, you don’t want to unfairly add to their cognitive load by making them wonder why you used “item-writing” in one place and “item writing” in another. Was it on purpose? Is there a hidden meaning that they need to pick up on? Does the hyphenated one have a different meaning? A nervous learner may obsess over meaningless things like this, wasting time and preventing them from showing you what they truly know.

If you’re writing items that use distractors (e.g., multiple-choice questions), make sure the distractors are plausible. An obviously wrong distractor is a wasted distractor that only helps the learner guess correctly, making it look like they know something they don’t. At the same time, make sure that you have only one truly correct choice and that you’re not tricking the learner. Also be careful about using keywords in the choice that are also used in the main body of the question, as this can clue the learner to the correct answer, or unfairly trick the learner into selecting the wrong answer.

Also, keep in mind that the verb you use can raise or lower the cognitive level of the question. A simple recall question would use verbs like define, list, or identify. You could take that item to the next level by using “interpretation” verbs such as differentiate, contrast, categorize and distinguish. An even higher cognitive level can be achieved by using “problem solving” verbs such as formulate, value, rate, revise and evaluate.

2 Responses to “Writing Test Items – Test Design and Delivery Part 4”

  1. Your last paragraph need some refinement. The verb is primarily irrelevant to the level of learning. The critical issue is the condition in the objective, e.g., is the performance that is required previously encountered or not? Repeating something word for word is Bloom level Knowledge. Recognizing a new (previously unencountered) instance moves it up a level quickly.

    I could ask the test taker to “identify ” the causes of a sales downturn. If these were provided in class, then its a memory level. If they are reading a case study then “list” may be at the Analysis level.

Leave a Reply