Guidelines and standards for defensible assessments

Posted by Greg Pope

I have been asked on occasion what guidelines and standards are available to ensure that an assessment program aligns with best practices and is defensible.

Organizations conducting assessments undertake internal reviews of where their assessment program is in relation to internationally recognized guidelines on assessment. High -stakes organizations will at times hire companies that specialize in psychometric audits to conduct a thorough review of the assessment processes and practices within the organization. This will usually yield an audit scorecard outlining where an organization is doing well and where is should improve according to the guidelines.

The most common guidelines document to be used for these sorts of audits is the “Standards for Educational and Psychological Testing” . This document is organized into numbered sections, each of which provides details on what is expected of an assessment program in various areas. For example, section 8.7 states:
“Test takers should be made aware that having someone else take the test for them, disclosing confidential test material, or any other form of cheating is inappropriate and that such behavior may result in sanctions.”

During an audit an organization may get scored on how it performs in relation to each section of the document with information on where the organization performed well and where the organization performed poorly and needs to improve. So in the example above, if an organization has a clearly written candidate agreement in place that participants need to agree to before continuing to the assessment, the organization would obtain full marks for this standard. If the organization does not have a candidate agreement in place, this would be an area for improvement.

One can go through the entire standards document and create a checklist to record the status of where the organization fairs compared to each of the standards, as in the example below:

Other guidelines and standards that you can use to help benchmark and improve your assessment program are:

• The Standards for the Accreditation of Certification Programs (NCCA)
• Conformity assessment – general requirements for bodies operating certification of persons 17024 (2003; ISO)
• Guidelines for Computer-Based Testing from The Association of TestPublishers
• International Test Commission (ITC) Projects:
  • ITC Guidelines on Adapting Tests
  • ITC Guidelines on Test Use
  • ITC International Guidelines on Computer-Based and Internet-Delivered Testing
• Principles for Fair Student Assessment Practices for Education in Canada
• Code of Fair Testing Practices in Education – American Psychological  Association
• NCTA Professional Standards and Guidelines for Post-Secondary Test Centers
• Information technology — A code of practice for the use of information technology (IT) in the delivery of assessments (ISO/IEC 23988:2007)
• Standards for Teacher Competence in Educational Assessment of Students (1990)
• Guidelines for Computerized Adaptive Test Development and Use in Education (1995) (See www.acenet.edu).
• The Regulatory Principles for e-assessment developed by the Qualification and Curriculum Authority (QCA)

I hope this article was helpful!