Posted by Julie Delazyn
Creating strong, defensible assessments is a frequent theme in this blog.
Most recently, Doug Peterson took readers step by step through a ten-part series of posts on Test Design and Delivery. Based on this series, he put together a presentation (now available on SlideShare) called Five Steps to Better Tests: Best Practices for Design and Delivery for the Questionmark Users Conference in March.
The presentation goes through each key step test creation process from planning the test, with tips like avoiding bias and stereotyping, to creating it and setting passing standards, delivering it while ensuring test security and finally, evaluating it using item-level data to improve item quality.
Enjoy the presentation below, and mark your calendar for the 2014 Questionmark Users Conference, March 4-7 in in San Antonio, Texas.
Posted by Julie Delazyn
Last week I wrote about deploying compliance-related assessments, as part of a series of posts offering good practice recommendations from our white paper, The Role of Assessments in Mitigating Risk for Financial Services Organizations.
This paper describes five stages of deploying legally defensible assessments, along with specific recommendations for people in different job roles. Some of these recommendations are specific to Questionmark technologies, but most can be applied to any testing and assessment system.
The five stages:
Today, let’s look at good practice for the third stage: authoring. You will find more recommendations in the White Paper:
Sharon Shrock & William Coscarelli’s Criterion-Referenced Test Development: Technical and Legal Guidelines for Corporate Training provides actionable, practical advice on test development. Sharon and Bill will conduct a workshop on writing valid, reliable tests in Baltimore on Sunday, March 3. Participants will explore testing best practices and will learn how to meet rigorous competency testing standards.
You can register for this workshop when you register for the Questionmark Users Conference or add the workshop later. It’s up to you!
Validity and reliability, fairness and cut scores are key concerns when it comes to the legal defensibility of assessments. Security is another important element, since cheating and security breaches can throw the validity and fairness of a test into question.
Our white paper, Defensible Assessments: What You Need to Know, aims to help readers determine what defensibility means, to inform them about standards and best practices and to describe how to ensure and evaluate defensibility.
Here are few quick security tips from the paper, which you are welcome to download with our compliments:
- Assign role-based security rights so that access to item banks is limited
- Determine the topic and assessment folders that authors may access and define how they are allowed to use them. For instance, should a person be able only to view content or should they also be allowed to edit it?
- Ensure that the right people are taking a test by requiring participants to log in with a username and password
- Set limits on the number of attempts according to the stakes of the assessment
- Establish specific dates and times for taking a test
- Organize participants into groups and hierarchical subgroups — each with designated default test centers — for simple and secure assessment scheduling
- Maintain up-to-date information about participants to avoid confusion in test scheduling and the processing of test results