New white paper: Questionmark and Microsoft Office 365

Posted by John Kleeman

I’m pleased to inform you of a new white paper fresh off the press on Questionmark and Microsoft Office 365.

Office logoThis white paper explains how Microsoft Office 365 complements the Questionmark OnDemand assessment management system; and how you can use Office 365 to launch Questionmark surveys, quizzes, tests and exams, how to consume Office 365 resources within Questionmark, and how Office 365 can help analyze results from assessments. You can download the white paper here.

The white paper also describes some of the reasons that organizations use assessments and why it is important for assessments to be valid, reliable and trustable.

Launching assessments from Office 365

Being able to call assessments from within Office 365 allows you to closely connect an assessment to content, for example to check understanding after learning. The white paper describes how you can:

  • Call Questionmark assessments from the Office 365 app launcher
  • Launch an assessment from within a Word, Excel or other Office document
  • Embed an assessment inside a PowerPoint presentation
  • Launch or embed assessments from SharePoint
  • Use SAML to have common identities and seamless authentication between Office 365 and Questionmark OnDemand. The benefit of this is that test-takers can login once to Office 365 and then can take tests in Questionmark OnDemand without needing to login again.

Using Office 365 resources within assessments

Illustration of a picture of a video being used inside an assessmentAssessments of competence are in general more accurate when the questions simulate the performance environment being measured. By putting video, sound, graphics and other media within question stimulus, you help put the participant’s mind into an environment closer to how he/she will be when doing a real-world job task. This makes the question more accurate in measuring the performance of such tasks.

To help take advantage of this, a common use of Office 365 with Questionmark OnDemand is  to make media and other resources that you can use within assessments. The white paper describes how you can use Office 365 Video, PowerPoint, SmartArt and other Office 365 tools to make videos and other useful question content.

Using Office 365 to help analyze results of assessments

People have been using Microsoft Excel to help analyze assessment results since the 1980s and the white paper describes some suggestions on how to do that most effectively with Questionmark OnDemand.

Newer Microsoft tools can also be used to provide powerful insight into assessment results. Questionmark OnDemand makes available assessment data in an OData feed, which can be consumed by business intelligence systems like Power BI. OData is an open protocol to allow the creation and consumption of queryable and interoperable data in a simple and standard way. The white paper also describes how to use OData and Power BI to get further analysis and visualizations from Questionmark OnDemand.

 

The white paper is easy to read and gives practical advice. I recommend reading this white paper if your organization uses Office 365 and Questionmark or if you are considering doing so. You can download the white paper (free with registration) from the Questionmark website.  You can also see other white papers, eBooks and other helpful resources at www.questionmark.com/learningresources.

 

How many test or exam retakes should you allow? Part 2

John Kleeman HeadshotPosted by John Kleeman

In my last post, I offered some ideas about what to consider when determining your retake policy regarding a certification assessment measuring competence and mastery. Some of the issues to balance are test security, fairness, a delay between retakes and the impact of retakes on test preparation. In this conclusion to the post, I’ll share what a few other organizations do and how you might approach deciding the number of retakes to allow.

Here is how a few respected certification programmes manage retakes

SAP have the following rules in their certification programme:

No candidate may participate in the same examination for the same release more than three times. A candidate who has failed at an examination three times for a release may not attempt that examination again until the next release. 

Microsoft allow up to 5 attempts in a 12-month period and then impose a 12-month waiting period. They also have gaps of several days between retakes, with the number of days increasing for subsequent retakes.

The US financial regulator FINRA requires a waiting time of 30 days between exams, but if you fail an exam three or more times in succession, you must wait 6 months before taking it again.

What’s the right answer for you?

The right answer depends on your circumstances. Many programmes allow retakes but have rules in place to limit the delivery rate of the assessment in order to limit content exposure.

1. You should communicate your retake policy to participants and to stakeholders who see the results of the assessments.

2. If you release scores, you also need to decide whether you will have a policy  as to whether scores for all attempts are released, or (as many organizations do) only for the successful attempt. Section 11.2 of the the Standards for Educational and Psychological Testing states

“Test users or the sponsoring agency should explain to test takers their opportunities, if any, to retake an examination; users should also indicate whether the earlier as well as later scores will be reported to those entitled to receive score reports.”

3. You should not allow people to retake a test they have passed.

4. You should consider requiring a period of time to elapse before someone retakes an exam if they fail. This allows time for them to update their learning. You can easily set this up when scheduling within Questionmark software, for example the dialog below gives a 7-day gap.

Limit days between retakes

5. Unless special circumstances apply, you will usually want to allow at least one retake and probably at least two retakes.

6. You may want to consider some intervention or stop procedure after a certain number of failed attempts. A common number I’ve heard anecdotally is three attempts, but it will depend on each assessment program’s own individual factors and use cases.  If this is an internal compliance exam, you might want to organize some remedial training or job review. If this is a public exam, you might want to ensure a longer time period to allow reflection and re-learning.

Please feel free to comment below if you have alternative thoughts on the number of retakes to allow.

Building on SharePoint for your learning infrastructure : A SlideShare presentation

Headshot JuliePosted by Julie Delazyn

Social networking, wikis, blogs, portals and collaboration tools play an increasingly important role, offering powerful ways to increase participation and sustain momentum in learning. Enterprise portal applications such as Microsoft SharePoint offer content management and facilitate information sharing across boundaries.

According to Bill Finegan, Vice President of Enterprise Technology Solutions at GP Strategies Corporation, the “portalization” of learning and development enables workers to gain the knowledge and skills they need to succeed, using technologies they engage with every day. How can tools like these fit in with your organization’s learning needs? And how can you make them work effectively together?

Bill answered these and other questions during a presentation called Get to the Point! Leverage SharePoint to meet your Learning and Development Needs at the 2013 Questionmark Users Conference,

Now available on SlideShare, this presentation explores the elements of a dynamic learning ecosystem and explains out how to combine SharePoint, Questionmark and other technologies to provide a learning environment suitable for today’s workers, This is just one example of what people learn about at our Users Conferences. Registration is already open for the 2014 Users Conference March 4 – 7 at the Grand Hyatt on the beautiful Riverwalk in San Antonio, Texas. Plan to be there!