Sharon Shrock
Posted by Joan Phaup
A highlight of the Questionmark Users Conference in Memphis was Sharon Shrock and Bill Coscarelli’s keynote, “Results You Can Rely On: What We’ve Learned from 25 Years of Criterion-Referenced Measurement.”
Sharon and Bill took us from the early history of testing to the work of Robert Glaser, who in laying the foundations of criterion reference testing focused on the importance of setting objectives and measuring test takers against a standard instead of against one another. They had us try William Angoff’s method of setting cut scores ( click here to access a detailed paper on this subject by Bill and Sharon together with Andrew Barrett and John Kleeman) and reviewed Donald Kirkpatrick’s four levels (“Don’t skip Kirkpatrick Level 2 if you are using 3 and 4!”) . They also described the
Bill Coscarelli
six levels of what they call the “Certification Suite” (noted by Dr. Will Thalheimer in his review of the latest edition of Shrock and Coscarelli’s book, Criterion-Referenced Test Development.)
A key point was that people on a job are doing much more than remembering facts. So test questions, rather than just operating at the memory level, should deal with real problems that people who do a particular job have to think through.
Tuesday’s conference schedule also included sessions on how to organize item banks, effective reporting techniques, teaching faculty to use Questionmark Perception, item analysis, and test maintenance best practices.
Wednesday brought some encore tech training sessions and a “road ahead” session seeking participants’ reactions to ideas for future features and services. Then some quick good-byes as participants headed for home. Here’s looking forward to next year’s conference!
Posted by Joan Phaup
