Create high-quality assessments: Join a March 4 workshop

Joan Phaup 2013 (3)

Posted by Joan Phaup

There will be a whole lot of learning going on in San Antonio March 4 during three workshops preceding the Questionmark 2014 Users Conference.

These sessions cover a broad range of experience levels — from people who are just beginning to use Questionmark technologies to those who want to understand best practices in test development and item writing.

Rick Ault

Rick Ault

Questionmark Boot Camp: Basic Training for Beginners (9 a.m. – 4 p.m.)

Questionmark Trainer Rick Ault will lead this hands-on workshop, which begins with a broad introduction to the Questionmark platform and then becomes an interactive, hands-on practice session. Bring your own laptop to get some firsthand experience creating and scheduling assessments. Participants will also get acquainted with reports and analytics.

Melissa Fein web

Dr. Melissa Fein

Test Development Fundamentals (9 a.m. – 12 p.m.)

Whether you are involved in workplace testing, training program evaluation, certification & certificate program development, or academic testing, an understanding of criterion-referenced test development will strengthen your testing program. Dr. Melissa Fein, author of Test Development Fundamentals for Certification and Evaluation, leads this morning workshop, which will help participants judge test quality, set mastery cutoff points, and improve test quality.


Mary Lorenz

The Art and Craft of Item Writing (1 p.m. – 4 p.m.)

Writing high-quality multiple-choice questions can present many challenges and pitfalls. Longtime educator and test author Mary Lorenz will coach workshop participants through the process of constructing well-written items that measure given objectives. Bring items of your own and sharpen them up during this interactive afternoon session.


Choose between the full-day workshop and one or both of the half-day workshops.

Conference attendees qualify for special workshop registration rates, and there’s a discount for attending both half-day sessions.

Click here for details and registration.




Workshop on Test Development Fundamentals: Q&A with Melissa Fein

Posted by Joan PhaupJoan Phaup 2013 (3)

We will be packing three days of intensive learning and networking into the Questionmark 2014 Users Conference in San Antonio March 4 – 7.

From Bryan Chapman’s keynote on Transforming Open Data into Meaning and Action to case studies, best practice advice, discussions, demos and instruction in the use of Questionmark technologies, there will be plenty to learn!

Even before the conference starts, some delegates will be immersed in pre-conference workshops. This year we’re offering one  full-day workshop and two half-day workshops.

Here’s the line-up:

Today’s conversation is with Melissa Fein, an industrial-organizational psychology consultant and the author of Test Development:  Fundamentals for Certification and Evaluation.

Melissa’s workshop will help participants create effective criterion-reference tests (CRT). It’s designed for people involved in everything from workplace testing and training program evaluation to certifications and academic testing.

What would you say is the most prevalent misconception about CRT?
…that a passing score should be 70 percent. The cutoff for passing might end up to be 70 percent, but that needs to be determined through a standard-setting process. Often people decide on 70 percent because 70 percent is traditional.

What is most important thing to understand about CRT?
It’s crucial to understand how to produce and interpret scores in a way that is fair to all examinees and to those who interpret and use the scores in making decisions, such as hiring people, promoting people, and awarding grades. Scores are imperfect by nature; they will never be perfect. Our goal is to produce quality scores given the limitations that we face.

How does CRT differ in the worlds of workplace testing, training, certification and academic assessment?
The process used to identify testing objectives differs for these different contexts.  However, there are more similarities than differencesin developing CRTs for workplace testing, training, certification and academic assessment.  The principles underlying the construction of quality assessments — such as validity, reliability, and standard setting — don’t differ.

When is CRT the most appropriate choice, as opposed to norm-referenced testing?
Anytime test scores are being compared to a standard, you want to use criterion-referenced testing. With norm referenced tests, you just want to compare one examinee’s scores with another. If you had police officers who have to pass fitness standards — maybe they have to run a mile in a certain amount of time – you would use CRT. But if the officers are running a benefit 5K race, that’s norm-referenced. You just want to find out who comes in first, second and third.

I understand you will be covering testing enigmas during the workshop. What do you have in mind?
Testing enigmas reflect best practices that seem to defy common sense until you look more closely. The biggest enigma occurs in standard setting. When most people think of setting standards for certifications, they like to think of a maximally proficient person. When I ask them to think of a minimally competent person, they think I’m pulling the rug out from under them! But in standard setting, you are trying to determine the difference between passing and failing, so you are looking to identify the minimally competent person: you want to define the line that distinguishes the minimally competent person from someone who is not competent.

What do you hope people will take away from their morning with you?
I hope people will walk away with at least one new idea that they can apply to their testing program. I also hope that they walk away knowing that something they are already doing is a good idea – that the workshop validates something they are doing in their test development work. Sometimes we don’t know why we do certain thing, so it’s good to get some reassurance.

Click here to read a conversation with Rick Ault about Boot Camp. My next post will be a Q&A with item writing workshop instructor Mary Lorenz.

You will save $100 if you register for the conference by January 30th. You can add a workshop to your conference registration or choose your workshop later.

Authoring compliance-related assessments: good practice recommendations

Headshot JuliePosted by Julie Delazyn

Last week I wrote about deploying compliance-related assessments, as part of a series of posts offering good practice recommendations from our white paper, The Role of Assessments in Mitigating Risk for Financial Services Organizations.

This paper describes five stages of deploying legally defensible assessments, along with specific recommendations for people in different job roles. Some of these recommendations are specific to Questionmark technologies, but most can be applied to any testing and assessment system.

The five stages:

Compliance five steps
Today, let’s look at good practice for the third stage: authoring. You will find more recommendations in the White Paper:

authoring chart

Sharon Shrock & William Coscarelli’s Criterion-Referenced Test Development: Technical and Legal Guidelines for Corporate Training provides actionable, practical advice on test development. Sharon and Bill will conduct a workshop on writing valid, reliable tests in Baltimore on Sunday, March 3. Participants will explore testing best practices and will learn how to meet rigorous competency testing standards.

You can register for this workshop when you register for the Questionmark Users Conference or add the workshop later. It’s up to you!

Full house at learning event in New Zealand

rafael-conf-australia2Posted by Rafael Lami Dozo

img00113-20090811-2230-2I posted recently about an upcoming  three-day learning event in Auckland, New Zealand, focusing on assessment best practices. Now I’d like to update you on the great turnout, exciting customers, and the full house that participated in the workshop!

The Online Assessments Symposium organized by Business Toolbox was packed with learning opportunities: the first two days were devoted to instruction on best practices img00114-20090811-2231-3in creating assessments, and the third brought together industry experts to share advice about moving assessments online.

It was motivating to see academic and corporate Questionmark users sharing  their experiences in successfully implementing assessments and enjoying some impressive case study presentations. I took img00117-20090811-2231-2these photos with my BlackBerry to give you a sense of the group that gathered.

We will continue to perform workshops like this one around the world so stay tuned for our next location.

Three-Day Symposium in New Zealand: Focus on Assessment Best Practices

rafael-conf-australia2Posted by Rafael Lami Dozo

I am looking forward to an exciting three-day learning event in Auckland, New Zealand, an Online Assessments Symposium organized by Business Toolbox. The first two days will be devoted to instruction on best practices in creating assessments, and the third will bring together industry experts to share advice about moving assessments online.


Participants take a break from learning at the Online Assessments Symposium in Colombia last month

Last month I presented a similar program in Colombia, and the response was tremendous. A good number or people from both New Zealand and Australia have already registered for the Auckland symposium, and we still have room for a few more. You may opt to register for one or both workshops, but registration will end soon so I encourage you to act now if you would like to join us. Not only is this a great learning opportunity: it also will give you the chance to network with other learning and assessment professionals in the region.

Here are some of the activities we have planned:

August 10 and 11 (Monday and Tuesday) will feature an intensive 2-day workshop: Creating Assessments that Get Results. This course is designed to instruct and engage you in applying best practices for online assessment design, development and delivery.

On August 12 (Wednesday) we will hold a seminar on Putting Your Assessments Online. The seminar  will feature an excellent line-up of industry experts and practitioners who will talk about assessment best practice and their own experiences with moving from paper-based to online assessments.

This is a great opportunity for you to see examples of what other organizations are doing, and network with your industry peers.  I hope to see many of you at the symposium!

Please feel free to post your comments and questions below or you can contact me directly.