This is the second of a series of blog posts on assessment standards. Today I’d like to focus on the IMS QTI (Question and Test Interoperability) Specification.

It’s worth mentioning the difference between Specifications and Standards: Specifications are documents that industry bodies have agreed on (like IMS QTI XML), while Standards have been published and committed to by a formal legal body (like AICC or HTML). A Specification is less formal than a Standard but still can be very useful for interoperability.

Questionmark was one of the originators of QTI. When we migrated our assessment platform from Windows to the Web in the 1990s, our customers had to migrate their questions from one platform to the other. As you will know, it takes a lot of time to write high quality questions, and so it’s important to be able to carry them forward independently of technology. We knew that we’d be improving our software over the years and we wanted to ensure the easy transfer of questions from one version to the next. So we came up with QML (Question Markup Language), an open and platform-independent method of maintaining questions that makes it easy for customers to move forward in the future.

Although QML did solve the problem of moving questions between Questionmark versions, we met many customers who had difficulty bringing content created in another vendor’s proprietary format  into Questionmark. We  wanted to help them, and we also wanted to embrace openness and allow Questionmark customers to export out their questions in a standard format if they ever wanted to leave us. So we worked with other vendors within the umbrella of the IMS Global Learning Consortium to come up with QTI XML, a language that describes questions in a technology-neutral way.  I was involved in the work defining IMS QTI as were several of my colleagues: Paul Roberts did a lot of technical design, Eric Shepherd led the IMS working group that made QTI version 1, and Steve Lay (before joining Questionmark) led the version 2 project.

Here is a fragment of QTI XML and you can see that it is a just-about-human-readable way of describing a question.

As readers of this blog will know, QMWISe is the name given to the set of web-services provided by Questionmark for integrating third-party applications with the Perception assessment management system. Used in conjunction with the Perception Integration Protocol (PIP), QMWISe is the starting point for developing connectors that integrate Perception with other applications like learning management systems.

The scope of QMWISe has grown over the years as John Kleeman reported in “Seven years of web services for easier integration”.  With over 100 methods to choose from, knowing which methods are best for your integration project can be a challenge.

Developers often work best by copying real examples of an interface in action.  It makes perfect sense for us to release the code for the connectors we develop ourselves in source code form to help supplement the simpler code examples available in the reference guide.

So I’m pleased to announce that we now have a new connector to integrate Perception with Blackboard Learn 9 and that this connector is available as a Community Edition under an open source license. The connector is a Blackboard building block and uses QMWISe and PIP to allow Perception tests to be scheduled within Blackboard courses.

The motto of the open source community is “release early, release often” and we’re capturing that spirit with the concept of a Community Edition.  A Community Edition is an important milestone on the way to developing full support for an integration product, however, it doesn’t represent supported software itself!  The Community Editions are aimed at a more technical audience who want to test out the latest version of the software before it is available as a supported release from the Questionmark website. We’ll be making updated versions of Community Edition software available on a regular basis as the development projects that underpin them progress.

The new Blackboard connector is not just available free of charge under an open source license. We’re also opening up the development process itself by hosting the project at OSCELOT, a community of developers dedicated to creating and maintaining open source software in the e-Learning community.  You can see how the project is progressing and download the latest version yourself from the Blackboard Connector page on our new developer support site.

And we’re not going to stop at Blackboard!  Look out for Community Editions of other integration connectors coming soon….

Posted by Tom King

Sometimes it’s important to know when someone finishes an assessment. Using standard, built-in Perception functionality you can have an automated email pushed to anyone you like–for instance the participant, a manager, or an instructor. But today I’d like to show how you can push an instant notification to an iPhone.

Perception PIP files can be a powerful yet lightweight way to integrate with web applications. Using only a PIP file and a $2.99 iPhone application, you can send push notifications of assessment completion or results directly from Perception to your iPhone (actually up to 5 iPhones or iPod Touch devices at once). Interested to see it in action? Take a look at this 45 second video.

It was surprisingly easy to do that with the help of the Perception Web Deployment Guide section on PIP files, and the Prowl API information. Of course, I also had to spend $2.99 to get Prowl on the iTunes App Store. If you’d like to learn how it is done, please take a quick look at the 5 minute explanation below.

To make things still easier, the Questionmark documentation team added a Knowledge Base article to the Questionmark Support site, How can I use PIP and Prowl to send push notifications to my iPhone? (Community logon by Questionmark Software Support Plan customers required). I hope this inspires you to do your own creative PIP file integrations. If you’ve already done something clever with PIP file integrations, please let me know at tom.king@questionmark.com.

The European Users Conference is less than one month away, and I’m pleased to announce that Tom King, Questionmark’s Interoperability Evangelist, and David Sloan from the University of Dundee will be leading the Tuesday morning General Session, focusing on E-Assessment and Interoperability, Standards and Accessibility.

Tom King is actively involved with many e-learning technology specification groups, and a regular contributor to this blog. Tom will provide an overview of the current status of major standards and the specification organisations behind them, and highlight some of the emerging needs and promising developments.  David Sloan will give an overview of accessibility-related legislation, standards and best practice, and show how Questionmark can help support the creation of accessible assessments.

The conference is set to be an exciting two days for Perception Users with Best Practice sessions on the latest trends in assessment management, eight Case Study presentations, and some great Technical Training sessions. Make sure you check out the full conference agenda and if you haven’t already done so, register for the conference!

I prepared a new segment on Understanding eLearning Standards. This segment addresses the “how” of elearning standards, and specifically run-time communication using the common AICC HACP specification. [Don't worry SCORM fans, there will be another segment focusing on the SCORM runtime.]

Standards fans (and hockey fans) are likely to appreciate the analogies used to explain a run-time environment in general. The video also steps through the lifecycle of an activity running in an LMS environment. Then I drill down to the specific of AICC, including both the common browser-to-LMS and the compelling server-to-server uses of AICC HACP.

Finally, the segment closes with a review of key resources from the AICC web site to help you make the most of AICC HACP.

By the way, here is an extra resource for members of the Questionmark Software Support Plan Community. There is a great Knowledge Base article on customizing the Perception v4 PIP file for AICC. This article shows how you can use a custom PIP file to utilize additional demographic or custom variables from an AICC compatible LMS. Check it out.

Stay tuned to the Questionmark Blog for the next segment that will address SCORM Run-Time Communication.

I’ve prepared a video podcast which is your introduction to key interoperability standards for elearning. It also serves as my introduction to video podcasts. Your feedback on both the content and the style will be put to use as I continue the series—so please post comments or send email.

The video for Part 1 provides a quick overview of the need for interoperability standards, the names of the keys standards, and the types of interoperability they support. Part 1 addresses AICC, ADL SCORM, IEEE LTSC and IMS specifications at a high level. It introduces the concepts of run-time communication, content packaging, and meta-data.

I hope you find it a good refresher if you are already somewhat knowledgeable about these standards, and an excellent introduction if you are new to most of this.

Dr.  Will Thalheimer of Work-Learning Research spoke with me recently about the role feedback plays in assessments and how it can be used to help learners.

Our conversation touches on the basics of using feedback effectively; if you want to learn more about this subject I recommend you check out Will’s research-to-practice paper: Providing Learners with Feedback. The paper examines the latest research on this complex topic and provides practical recommendations. You can find it in the Work-Learning Research Catalog or Questionmark’s white paper list.

So here’s Will! I hope you enjoy our conversation.

Posted by John Kleeman

A key objective for Questionmark Perception has been to make it an open system that handles integrations easily. Assessment isn’t usually standalone; most organizations need to integrate it with other organizational systems. There are many ways to integrate with Perception, including via our support of standards such as AICC, HR-XML and SCORM, but where standards are not available we recommend integration via our QMWISe web services.

Although web services are routine today, Questionmark adopted them very early: June 6th, 2009, marks the 7th anniversary of Questionmark’s web services, which we call QMWISe. (See our 2002 press release here.)

Two great advantages of web services are that you can call them from almost any platform or system and they are independent of the technology used. So you can code web services in almost any programming language or environment and interface with Questionmark Perception.

Another beauty of web services is that code written back in 2002 will still work in 2009,and code written today should still work in 2016! In the last seven years, there have been very substantial changes to the Questionmark Perception database format and to the user interfaces, but the APIs (Application Programming Interfaces) remain the same. And exactly the same code written then to call QMWISe will still work now. We have ambitious plans to continue developing Questionmark software in new ways, but code our customers write today for QMWISe will still work in the future.

Back in 2002, there were 37 web services methods. Over the years, we’ve added lots more methods and there are now 109. Example web services methods are to create a participant, schedule a participant or give a URL to get access to an assessment.

Many of our customers use QMWISe to integrate with Perception, so that as Perception versions change, their code can remain safe. We or our partners have also used QMWISe to build connectors to many other systems, including Blackboard, Moodle and uPortal. We also call QMWISe within our own software. For instance, Questionmark to Go passes all its results back via web services, and in the future we’ll be trying to use QMWISe more within other code–to “eat our own dog food” and ensure that QMWISe is fully able to be mission critical. By using web services within our own code, we will be driving QMWISe forward to cover more capabilities and so open up the platform to support a wide range of solutions integrated with third party applications.

One key lesson that we’ve learned over time with web services is that commitment and continuity are vital. No one wants to interface with a system that will change. And you need to have good documentation with examples, good scalability and good diagnostics–for instance a log of all SOAP traffic. We recommend that other developers consider making web services available from their own systems: it’s an excellent way of integrating.

In the future we’ll be announcing further improvements to QMWISe that should make it more useful for developers and provide easier ways for customers to integrate with Perception. Questionmark strongly recommends that anyone developing integration into our software uses our web services. We welcome questions, comments and suggestions for improvements, so let us know what you think!

Posted by Jim Farrell

Questionmark Live is our new subject matter expert (SME) web-based authoring tool that allows you to harvest content from SMEs and import it into Questionmark Perception.

So what does that mean to you and your organization? Think about how you are currently getting information from your SMEs. Is it always timely? Is it in a format that is easy for you to use? Our aim in creating this content harvesting tool was to help SMEs create items quickly and easily.

• Create 5 different question types
• Use a multi-lingual interface
• Create an unlimited number of questions
• Provide choice based feedback
• Add images and links to questions
• Download or email questions to a Perception user

Our tutorial will show you how easy it is to use this new tool, which we are providing  this new tool without charge to active Questionmark Software Support Plan customers. (Other Questionmark Communities members can request a free 30-day trial.)

Learn more about Questionmark Live.

Access Questionmark Live or request a 30-day trial. (You may be prompted for your Questionmark Communities password.)