Preparing to Create the Assessment – Test Design & Delivery Part 6
Posted By Doug Peterson
At this point in the design process, you’ve written all the items for your assessment. Before you assemble them into a test, they need to be reviewed. Be sure to link each item to the Test Content Outline (TCO), then ask a group of subject matter experts (SMEs) to review the questions. This very well could be the same group of SMEs that wrote the questions in the first place, in which case they can simply review each other’s work. There are three main things to look at when reviewing each item:
- Spelling and grammar
- Clarity – is it clear what the item is asking? Is the item asking only one question and does it have only one correct answer? Is the item free of any extraneous information, bias, and stereotyping?
- Connection to TCO – it is legitimate to include this item on this assessment because it clearly and directly pertains to the goals of the training.
Once you are confident that you have a complete set of well-written items that tie directly to your TCO, it’s time to start putting the assessment together. In addition to determining which questions from your item bank you want to include (which will be discussed in the next entry in this series), you must also develop test directions for the participant. These directions should include:
- Purpose of the assessment
- Amount of time allowed
- Procedures for asking questions
- Procedures for completing the assessment
- Procedures for returning test materials
As part of your participant directions, you may want to consider including sample items, especially if the format is unusual or unfamiliar to the participants. Sample items also help reduce test anxiety. Remember, you want to assess the participant’s true knowledge, which means you don’t want a “stress barrier” getting in the way.
In addition to the participant’s instructions, you also want to put together instructions for the assessment administrator – the instructor or proctor who will be handing the test out and watching over the room while the participant’s take the assessment. Having a set of written instructions will help ensure consistency when the assessment is given by different administrators in different locations. The instructions should include:
- The participant’s instructions, which should be read aloud
- How to handle and document irregularities
- The administrator’s monitoring responsibilities and methods (e.g., no phone conversations, walk around the room every 10 minutes, etc.)
- Hardware and software requirements and instructions, if applicable
- Contact information for technical help
As you develop your assessment, make sure that you are taking into account any local or national laws. For example, American test centers must comply with the Americans with Disabilities Act (ADA). The ADA requires that the test site be accessible to participants in wheelchairs and that compensation be made for certain impaired abilities (e.g., larger print or a screen reader for visually impaired participants). The administrator’s instructions should cover what to do in each case.