Multilingual Approach Includes Videos

Julie Delazyn HeadshotPosted by Julie Delazyn

Questionmark customers are spread across the globe, and so it’s important for us that our product is multilingual. Here are some of the features available:

Now, we’ve added an extra resource: two new playlists on Questionmark’s YouTube channel, which feature testimonials, tutorials, overview videos and how-to’s in both Portuguese and Spanish:

Browse, watch and enjoy!

Questionmark en Espanol

Questionmark em Português

 

New tools for building questions and assessments

Jim Farrell HeadshotPosted by Jim Farrell

If you are a Questionmark customer and aren’t using Questionmark Live, what are you waiting for?

More than 2000 of our customers have started using Questionmark Live this year, so I think now is a good time to call out some of the features that are making it a vital part of their assessment development processes.

Let’s start with building questions. One new tool our customers are using is the ability to add notes to a question. This allows reviewers to open questions and leave comments for content developers without changing the version of the question.

image 1

Now over to the assessment-building side of things. Our new assessment interface allows users to add questions in many different ways including single questions, entire topics, and random pull from a topic. You can even prevent participants from seeing repeated of questions during retakes when pulling questions at random. Jump blocks allow you to shorten test time or redirect for extra questions to participants who obtain a certain score. You can also easily tag questions as demographic questions so they can be used as filters in our reporting and analytics tools.

image 2

We have also added a more robust outcome capabilities to give your test administrators new tools for controlling how assessments are completed and reported. You can have multiple outcomes for different score bands, but you can also make it so participants have to get certain scores on particular topics before they can pass a test. For example, suppose you are giving a test on Microsoft Office and you set a pass score at 80%. You probably want to make sure that your participants understand all the products and don’t bomb one of them. You can set a prerequisite for each topic at 80% to make sure participants have knowledge of all areas before passing. If someone gets a 100% on Word questions and 60% on Excel questions, they would not pass. Powerful outcome controls help ensure you are truly measuring the goals of your learning organization.

image 3

If you aren’t using Questionmark Live you are missing out, as we are releasing new functionality every month. Get access and start getting your subject matter experts to contribute to your item banks.

Field Test Studies: Taking your items for a test drive

Austin FosseyPosted by Austin Fossey

In large-scale assessment, a significant amount of work goes into writing items before a participant ever sees them. Items are drafted, edited, reviewed for accuracy, checked for bias, and usually rewritten several times before they are ready to be deployed. Despite all this work, a true test of an item’s performance will come when it is first delivered to participants.

Even though we work so hard to write high-quality items, some bad items may slip past our review committees. To be safe, most large-scale assessment programs will try out their items with a field test.

A field test delivers items to participants under the same conditions used in live testing, but the items do not count toward the participants’ scores. This allows test developers and psychometricians to harvest statistics that can be used in an item analysis to flag poorly performing items.

There are two methods for field testing items. The first method is to embed your new items into an assessment that is already operational. The field test items will not count against the participants’ scores, but the participants will not know which items are scored items and which items are field test items.

The second method is to give participants an assessment that includes only field test items. The participants will not receive a score at the end of the assessment since none of the items have yet been approved to be used for live scoring, though the form may be scored later once the final set of items has  been approved for operational use.

In their chapter in Educational Measurement (4 th ed.), Schmeiser and Welch explain that embedding the items into an operational assessment is generally preferred. When items are field tested in an operational assessment, participants are more motivated to perform well on the items. The item data are also collected while the operational assessment is being delivered, which can help improve the reliability of the item statistics.

When participants take an assessment that only consists of field test items, they may not be motivated to try as hard as they would in an operational assessment, especially if the assessment will not be scored. However, field testing a whole form’s worth of items will give you better content coverage with the items so that you have more items that can be reviewed in the item analysis. If field testing an entire form, Shmeiser and Welch suggest using twice as many items as you will need for the operational form. Many items may need to be discarded or rewritten as a result of the item analysis, so you want to make sure you will still have enough to build an operational form at the end of the process.

Since the value of field testing items is to collect item statistics, it is also important to make sure that a representative sample of participants responds to the field test items. If the sample of participant responses is too small or not representative, then the item statistics may not be generalizable to the entire population.

Questionmark’s authoring solutions allow test developers to field test items by setting the item’s status to “Experimental.” The item will still be scored, and the statistics will be
generated in the Item Analysis Report, but the item will not count toward the participant’s final score.

qm Properties

Setting an item’s status to “Experimental” in Questionmark Live so that it can be field tested.

Top 5 Questionmark Videos in 2013

Headshot JuliePosted by Julie Delazyn

As we near the end of the year, we’d like to highlight some of the most popular videos we’ve featured here on the blog in 2013.

We have been posting and sharing many videos from the Questionmark Learning Café. There you can find more than three dozen videos, demos and other resources on everything from quick tutorials to complete webinars about best practices in the use of online surveys, quizzes, tests and exams.

The five most popular videos in 2013… Drumroll, please…

5. Actionable Data And The A-Model
4. Introduction to Questionmark
3. Copy & paste & enhanced question selection in Questionmark Live
2. Assessments Through the Learning Process
1. How to author a Hot Spot Question using Questionmark Live

Thank you for watching, and look for more videos in 2014!

Questionmark Live: Watching the numbers grow

Jim Farrell HeadshotPosted by Jim Farrell

I hope everyone is having a wonderful holiday season.

Typically I come to you telling you about amazing new features, but this time I want to talk about the number of items created in Questionmark Live over the past year.

Let me set the stage. We started the year with approximately 30,000 questions being created in Questionmark Live each month.
As part of the team that gave birth to our newest authoring tool, I was over the moon about his strong start. This proved to me that the features we were releasing were helping people build up their item banks. I was excited enough as it was, but then came September, October and November.

September saw or most impressive increase in usage. More than 97,000 questions were created — almost 100,000 questions in just one month.

October followed with more than 74,000 questions –- including more than   20,000 questions on October 16th alone. This is the number our development teams were most excited about: 20,000 questions!  And the system ran flawlessly, speaking to the scalability of the software and the OnDemand Platform.

November proved that the usage was legitimate. More than 121,000 questions were created that month.

It’s clear now that Questionmark Live is the preferred authoring tool among Questionmark users — with ease of use and scalability as its foundation.

You didn’t think I would end without talking about something new, did you?

I can’t resist telling you that you no longer need to approve authors to have access to Questionmark Live. Anyone with a valid Questionmark Communities account can gain access to Questionmark Live. We hope this makes it easier for you to crowd- source content within your organization and write good questions to solve real business problems.

***

The Questionmark 2014 Users Conference will include bring-your-own-laptop sessions on Creating Items and Topics as well as Collaborative Assessment Authoring in Questionmark Live. The early-bird registration discount of $200 is available through tomorrow December 12th, so sign up now!

Winter Webinars in the UK: Questionmark Live updates and more

Posted by Chloe MendoncaChloe Mendonca

The holidays are just around the corner, but the excitement of learning continues year-round here at Questionmark!

Our free, one-hour web seminars give you the opportunity find out what’s happening in the world of online assessment and consider which tools and technologies would be most useful to you. Here’s the current line-up:

What’s New in Questionmark Live Browser-Based Authoring?

With the addition of two new item types to Questionmark Live, our browser-based authoring tool, we have two web seminars set up for you to learn about these and other changes you may have missed. Get an overview of Questionmark Live’s capabilities and all the latest new features.

  • Friday, 13th December 2013 – 3:00 PM (GMT – London)
  • Friday, 24th January 2014 – 3:00 PM (GMT – London)

Creating Assessments for Mobile Delivery

If you want some top tips on how to create assessments for mobile devices using Questionmark’s responsive design features, then don’t miss this webinar.

A report earlier this year stated that the number of smartphones, tablets, laptops and internet-capable devices would exceed the number of humans by the end of 2013 —  and that has come to pass. Mobile delivery is the way to reach people on the go. This session will help you get started.

  • Monday, 13th January 2014 – 2:00 PM (GMT – London)
  • Tuesday, 29th January 2014 – 10:30 AM (GMT – London)

Introduction to Questionmark’s Assessment Management System

For a more general overview on Questionmark’s assessment technologies join seminar, which explains and demonstrates key features and functions available in Questionmark OnDemand and Questionmark Perception. Spend an hour with a Questionmark expert learning the basics of authoring, delivering and reporting on surveys, quizzes, tests and exams.

  • Monday, 25th November 2013 – 2:30 PM (GMT – London)
  • Tuesday, 3rd December 2013 – 10:30 AM (GMT – London)
  • Thursday, 16th January 2014 – 11:00 AM (GMT – London)

Click here to choose your webinar and register online. I hope you enjoy the session you choose and if you have any questions,  reach out to us!