Best Practice Events Psychometrics Standards

Standard Setting: A Keystone to Legal Defensibility

Since the last Questionmark Users Conference, I have heard several clients discuss new measures at their companies requiring them to provide evidence of the legal defensibility of their assessment. Legal defensibility and validity are closely intertwined, but they are not synonymous. An assessment can be legally defensible, yet still have... Read More

Authoring Best Practice Events Psychometrics Standards

Item Development – Organizing a content review committee (Part 2)

Posted by Austin Fossey In my last post, I explained the function of a content review committee and the importance of having a systematic review process. Today I’ll provide some suggestions for how you can use the content review process to simultaneously collect content validity evidence without having to do a... Read More

Authoring Best Practice General How-To's Psychometrics Standards

Item Development – Five Tips for Organizing Your Drafting Process

Posted by Austin Fossey Once you’ve trained your item writers, they are ready to begin drafting items. But how should you manage this step of the item development process? There is an enormous amount of literature about item design and item writing techniques—which we will not cover in this series—but as... Read More

Authoring Best Practice General Innovative Technologies Psychometrics Writing Assessments

Item Development – Training Item Writers

Posted by Austin Fossey Once we have defined the purpose of the assessment, completed our domain analysis, and finalized a test blueprint, we might be eager to jump right in to item writing, but there is one important step to take before we begin: training! Unless you are writing the entire... Read More

Authoring Best Practice General Interoperability Psychometrics Standards

Item Development – Managing the Process for Large-Scale Assessments

Posted by Austin Fossey Whether you work with low-stakes assessments, small-scale classroom assessments or large-scale, high-stakes assessment, understanding and applying some basic principles of item development will greatly enhance the quality of your results. This is the first in a series of posts setting out item development steps that will help... Read More

Authoring Best Practice General Interoperability Psychometrics Standards

Trustworthy Assessment Results – A Question of Transparency

Posted by Austin Fossey Do you trust the results of your test? Like many questions in psychometrics, the answer is that it depends. Like the trust between two people, trustworthy assessment results have to be earned by the testing body. Many of us want to implicitly trust the testing body, be... Read More

Authoring Best Practice Innovative Technologies Psychometrics

Field Test Studies: Taking your items for a test drive

Posted by Austin Fossey In large-scale assessment, a significant amount of work goes into writing items before a participant ever sees them. Items are drafted, edited, reviewed for accuracy, checked for bias, and usually rewritten several times before they are ready to be deployed. Despite all this work, a true test... Read More

Best Practice General Interoperability Psychometrics Standards

Assessment Report Design: Reporting Multiple Chunks of Information

Posted by Austin Fossey We have discussed aspects of report design in previous posts, but I was recently asked whether an assessment report should report just one thing or multiple pieces of information. My response is that it depends on the intended use of the assessment results, but in general, I... Read More

Authoring General Psychometrics Standards

Discussing the revised Standards for Educational and Psychological Testing

Posted by Austin Fossey I just returned from the National Council for Measurement in Education (NCME) annual meeting in Philadelphia, which is held in conjunction with the American Educational Research Association (AERA) annual meeting. There were many big themes around research and advances in assessment, but there were also a lot... Read More