Posted by Howard Eisenberg
I had a typical education. I took lots of tests. Knowing what I know now about good testing practice, I wonder how many of those tests really provided an accurate measure of my knowledge.
Common testing practices often contradict what is considered best practice. This piece will focus on four of the most common “myths” or “mistakes” that teachers, subject matter experts, trainers and educators in general make when writing test questions.
1) A multiple choice question must have at least four choices. False.
Three to five choices is considered sufficient. Of course the fewer the choices, the greater the chance a test-taker can guess the correct answer. However, the point however is you don’t need four choices, and if you are faced with the decision of adding an implausible or nonsensical distracter to make four choices, it won’t add any measurement value to the question anyway. Might as well just leave it at three choices.
2) The use of “all of the above” as a choice in a multiple choice question is good practice. False.
It may be widely used but it is poor practice. “All of the above” is almost always the correct answer. Why else would it be there? It is tacked onto a multiple choice question so it can have only one best answer. After all, writing plausible distracters is difficult. If at least two of the other choices answer the question, then “all of the above” is the answer. No need to consider any more choices.
3) Starting a question with “Which of the following is not …” is considered best practice. False.
First, the use of negatives in test questions should be avoided (unless you are trying to measure a person’s verbal reasoning ability). Second, the use of the “which of the following …” form usually results in a question that only tests basic knowledge or recall of information presented in the text or in the lecture. You might as well be saying: “Which of the following sentences does not appear exactly as it did in the manual?
A) Copy > paste (from manual) choice 1
B) Copy > past choice 2
C) Copy > past choice 3
D) Make something up
While that may have some measurement value, my experience tells me that most test writers prefer to measure how well a person can apply knowledge to solve novel problems. This type of question just won’t reach that level of cognition. If you really want to get to problem-solving, consider using a real-world scenario and then posing a question.
4) To a subject matter expert, the correct answer to a good test question should be apparent. True.
A subject matter expert knows the content. A person who really knows the content should be able to identify the best answer almost immediately. Test writers often hold the misconception that a good test question is one that is tricky and confusing. No, that’s not the point of a test. The point is to attain an accurate measure of how well a person knows the subject matter or has mastered the domain. The question should not be written to trick the test-taker, let alone the expert. That just decreases the value of the measurement.
There are many more “do’s” and “don’ts” when it comes to writing good test questions. But you can start to improve your test questions now by considering these common misconceptions as you write your next test.