Practice versus perfection – 2014 Users Conference

Austin Fossey-42 Posted by Austin Fossey

The Questionmark team has just returned from the 2014 Users Conference, where we had a wonderful time showing off our latest work, discussing assessment strategies with our customers, and learning from each other in a great selection of seminars put on by both Questionmark staff and our clients.

At this year’s conference, I field tested two new presentations: Understanding Assessment Results and Principles of Psychometrics and Measurement Design. I got some great feedback from attendees so I can fine-tune them for the future, but these topics also started a lot of interesting conversations about what we as test developers would like to be doing and what we end up doing in practice.

A recurring theme of these conversations was that people felt there were occasionally aspects of their instruments that could be improved, especially in terms of capturing evidence for a measurement or supporting the validity of the results. In some cases they had an idea of what they wanted to improve, but they either did not know the test development methods they needed to apply, or they did not know how to convince their stakeholders and managers of the importance of specific initiatives. The concept of validity came up several times in these conversations—something we have touched on previously on this blog.

The ideals and realities of the assessment industry do not always align. For example, we may wish to do a construct validity study or an Angoff cut score meeting, but we may lack the resources, time, or stakeholder buy-in to engage in these activities.

I recognize how discouraging this can be for people who constantly want to improve the validity of their inferences, but I am excited to see so many people thinking critically about their assessment designs and searching for areas of improvement. Even if we cannot always implement every research study we are interested in, understanding the principles and best practices of good assessment design and interpretation can still guide our everyday work and help us to avoid invalid results. This blog is a good place to explore some of these principles, and so are Questionmark white papers and our Learning Café videos.

I look forward to continuing to work with (and learn from) our great client base throughout 2014 as we continue to advance our products. A special thanks to our attendees and presenters who joined us at the 2014 conference!

Create high-quality assessments: Join a March 4 workshop

Joan Phaup 2013 (3)

Posted by Joan Phaup

There will be a whole lot of learning going on in San Antonio March 4 during three workshops preceding the Questionmark 2014 Users Conference.

These sessions cover a broad range of experience levels — from people who are just beginning to use Questionmark technologies to those who want to understand best practices in test development and item writing.

Rick Ault

Rick Ault

Questionmark Boot Camp: Basic Training for Beginners (9 a.m. – 4 p.m.)

Questionmark Trainer Rick Ault will lead this hands-on workshop, which begins with a broad introduction to the Questionmark platform and then becomes an interactive, hands-on practice session. Bring your own laptop to get some firsthand experience creating and scheduling assessments. Participants will also get acquainted with reports and analytics.

Melissa Fein web

Dr. Melissa Fein

Test Development Fundamentals (9 a.m. – 12 p.m.)

Whether you are involved in workplace testing, training program evaluation, certification & certificate program development, or academic testing, an understanding of criterion-referenced test development will strengthen your testing program. Dr. Melissa Fein, author of Test Development Fundamentals for Certification and Evaluation, leads this morning workshop, which will help participants judge test quality, set mastery cutoff points, and improve test quality.


Mary Lorenz

The Art and Craft of Item Writing (1 p.m. – 4 p.m.)

Writing high-quality multiple-choice questions can present many challenges and pitfalls. Longtime educator and test author Mary Lorenz will coach workshop participants through the process of constructing well-written items that measure given objectives. Bring items of your own and sharpen them up during this interactive afternoon session.


Choose between the full-day workshop and one or both of the half-day workshops.

Conference attendees qualify for special workshop registration rates, and there’s a discount for attending both half-day sessions.

Click here for details and registration.




Top 5 Questionmark Videos in 2013

Headshot JuliePosted by Julie Delazyn

As we near the end of the year, we’d like to highlight some of the most popular videos we’ve featured here on the blog in 2013.

We have been posting and sharing many videos from the Questionmark Learning Café. There you can find more than three dozen videos, demos and other resources on everything from quick tutorials to complete webinars about best practices in the use of online surveys, quizzes, tests and exams.

The five most popular videos in 2013… Drumroll, please…

5. Actionable Data And The A-Model
4. Introduction to Questionmark
3. Copy & paste & enhanced question selection in Questionmark Live
2. Assessments Through the Learning Process
1. How to author a Hot Spot Question using Questionmark Live

Thank you for watching, and look for more videos in 2014!

Boot Camp for Beginners set for March 4, 2014

Joan Phaup 2013 (3)Posted by Joan Phaup

New customers who attended the Questionmark Users Conference in the past used to tell us that some hands-on instruction before the start of the conference would help them get a lot more out of the proceedings.

Enter Questionmark Boot Camp: Basic Training for Beginners – where people learn the basics before they join the throng at the conference. This full-day workshop has become a popular pre-conference option,  and we’re bringing it back again on Tuesday, March 4, 2014 – right before the Questionmark 2014 Users Conference in San Antonio.

I don’t get  to attend Boot Camp, but I did spend a few minutes talking about it with our trainer, Rick Ault, who has as much fun there as his pupils:

Rick Ault

Rick Ault

What happens at Boot Camp?
We talk about all the different Questionmark tools — how they are used together — and give people a solid understanding of the process and what they can do to build meaningful assessments. It’s hands-on. We ask people to bring their own laptops so that we can give them actual practice using the software. They have the chance to use Questionmark Live to build questions and assessments, then put that content onto a server to see it work.

Who should attend?
Any new users of Questionmark would benefit from it, because it’s designed to give people an understanding of what the product does and how it works.

What should they bring?
They should bring their laptops, plus some ideas for how they might like to use Questionmark. They should also bring some ideas for some fun content that they might want to create?

How does Boot Camp prepare new customers for the Users Conference?
It gives them exposure to all of the tools, and it helps them understand the process. By getting some hands-on experience with our technologies, they will be able to make better choices about what conference tracks and sessions to attend. They’ll also be able to think of meaningful questions to ask at the conference.

What do YOU like best about Boot Camp?
I like meeting new customers, and I like seeing their happiness when they create something. It’s great to see the birth of their new content as they join the Questionmark family!

Newcomers to Questionmark can join the familhy in style by attending Boot Camp. You can sign up when you register for the conference.