LTI certification and news from the IMS quarterly meeting

Steve Lay HeadshotPosted by Steve Lay

Earlier this month I travelled to Michigan for the IMS Global Learning Consortium’s quarterly meeting. The meeting was hosted at the University of Michigan, Ann Arbor, the home of “Dr Chuck”, the father of the IMS Learning Tools Interoperability (LTI) protocol.

I’m pleased to say that, while there, I put our own LTI Connector through the new conformance test suite and we have now been certified against the LTI 1.0 and 1.1 protocol versions.IMS

The new conformance tests re-enforce a subtle change in direction at IMS. For many years the specifications have focused on packaged content that can be moved from system to system. The certification process involved testing this content in its transportable form, matching the data against the format defined by the IMS data specifications. This model works well for checking that content *publishers* are playing by the rules, but it isn’t possible to check if a content player is working properly.

In contrast, the LTI protocol is not moving the content around but integrating and aggregating tools and content that run over the web. This shifts conformance from checking the format of transport packages to checking that online tools, content and the containers used to aggregate them (typically an LMS) are all adhering to the protocol. With a protocol it is much easier to check that both sides are playing by the rules  — so overall interoperability should improve.

In Michigan, the LTI team discussed the next steps with the protocol. Version 2 promises to be backwards-compatible but will also make it much easier to set up the trusted link between the tool consumer (e.g., your LMS) and the tool provider (e.g., Questionmark OnDemand).  IMS are also looking to expand the protocol to enable a deeper integration between the consumer and the provider. For example, the next revision of the protocol will make it easier for an LMS to make a copy of a course while retaining the details of any LTI-based integrations. They are also looking at improving the reporting of outcomes using a little-known part of the Question and Test Interoperability (QTI) specification called QTI Results Reporting.

After many years of being ‘on the shelf’ there is a renewed interest in the QTI specification in general. QTI has been incorporated into the Accessible Portable Item Protocol (APIP) specification that has been used by content publishers involved in the recent US Race to the Top Assessment Program. What does the future of QTI look like?  It is hard to tell at this early stage, but the buzzword in Michigan was definitely EPUB3.

Learning Tools Interoperability fulfills its promise!

Posted by Steve Lay

In previous blog posts I’ve discussed the new specification that the IMS Global Learning Consortium (IMS) has been working on called Learning Tools Interoperability or LTI for short.  See my first post on this subject here and the later update here.

In March, IMS released the final version of the specification.  This clears up any confusion between the earlier variants (Basic and Simple have been used as prefixes in the past) and sets a single standard for embedding tools in learning management systems (LMS) and portals.  The final specification also introduced an important new feature: a method of returning grade information from the tool to the LMS gradebook.

At Questionmark we have developed a number of connectors for integrating with popular learning management systems and portals, such as Blackboard and Moodle.  LTI provides us with an opportunity to replace those connectors with a unified approach to integration, so I can’t wait to get started with the new specification.

To this end we are now working on adding LTI support to our own software.  I recently attended an IMS workshop called “Creating Enterprise Aware, Multiplatform Apps with IMS Interoperability”.  At this workshop we heard about the latest developments in both IMS Common Cartridge and IMS LTI.  It was a great to meet some of the key people in the community and take a deep-dive on some of the technical details involved in implementing the specifications.

So how will Questionmark integrate using LTI?

In LTI terminology there are tool providers and tool consumers.  A tool consumer is typically an LMS or other type of portal that deals with user registration and assignment to courses where learning and assessment activities are aggregated.  A tool provider is a web-based service that provides a specialized experience to the learner such as an assessment.

Our first steps with LTI are aimed particularly at users of the Moodle LMS, though anyone with access to a web server running PHP and a suitable database will be able to integrate this way.  We’ve teamed up with an LTI specialist, Dr Stephen Vickers, to create an open source Community Edition connector that makes it easy for Moodle users to talk to Questionmark software using the new LTI protocol.  The project is hosted on the OSCELOT community development system.

This is just the beginning for Questionmark and LTI – so stay tuned for more updates on the role LTI will play in future Questionmark assessment technology solutions!

IMS specifications and standards update

Posted by Steve Lay

Last week I attended the IMS Global Learning Consortium‘s quarterly meeting in Nottingham, UK.  The meeting was co-located with JISC-CETIS, JISC is a collaboration of UK academic institutions and CETIS is their “Centre for Education Technology Interoperability Standards”. Essentially, CETIS helps promote the development and adoption of technical standards within the JISC community, while also playing a key role in advising JISC’s e-Learning programme.

The message from IMS was very clear. Three key standards provide a framework which covers the main interoperbility requirements of education. These standards are: Learner Tools Interoperability (LTI), Learner Information Services (LIS) and Common Cartridge. The first two are of particular interest.


I don’t think IMS has ever seen one of its specifications developed and adopted as rapidly as LTI. LTI allows a Learning Management System (LMS) or portal to be used to launch a wide range of activities hosted externally. In the past, content was either pre-loaded onto the LMS itself or hosted on an associated content server.  LTI is a simple mechanism that opens up the LMS to content hosted anywhere on the web using a simple extension to HTTP, the web protocol for accessing web pages.

When you click on a link to a website your browser navigates you there, but the  website knows little or nothing about where you linked to it from. When you click on an LTI link your browser does the same thing, but several important pieces of information are securely passed to the new website: your user identity, the context of the link you clicked (such as the course or even the course page from which you were coming) and your role within that context (such as instructor or student). This enables the remote content to behave as if it were a seamless part of the user’s learning experience. In fact, it is as if the content had been packaged up and hosted on the LMS itself.

LTI promises to open up the definition of ‘content’ to include a wider range of activities and tools, including assessments!


The related LIS specification enables the exchange of information about people involved in the learning experience. A tool launched from an LTI-enabled link can use LIS to find out more information about the user, or perhaps adjust the user’s learning record with updated test scores. If LTI is used to initiate the link between two systems it is LIS that is used to sustain it.

* *

In addition to the presentations from IMS, the conference also contained some interesting sessions from the JISC-CETIS community. This community has been very active in the development of Question and Test Interoperability (QTI). It appears that real progress is now being made with the demonstrators required by IMS before the latest draft can be promoted to a final specification.  Readers of this blog may feel that we’ve been here before, but there is reason to believe that the specification’s time finally has come. QTI forms an important foundation for the Accessible Portable Item Protocol (APIP) – a US-led accessibility initiative.

Conference Close-up: Technical Standards and Questionmark’s Open Assessment Platform

Posted by Joan Phaup

The program for the  Questionmark Users Conference is full of informative sessions about everything from item and tests analysis analytics to techniques for writing assessments measure skill and ability as well as knowledge. Questionmark Intetegration Team Lead Steve Lay will conduct a session on Technical Standards and Questionmark’s Open Assessment Platform.

Q: What’s your role at Questionmark?

Steve Lay

A: I’m the Integration Team Lead, so I look after all aspects of the product  concerned with integrating Questionmark with other systems — both within an organization and over the Internet.  I am Product Owner for our Connector products, which enable integrations of Questionmark Perception with other key systems such as Sharepoint Portal Server, Blackboard, Moodle and others. And I am heading our Open Assessment Platform Initiative.

Q: You just mentioned the Open Assessment Platform. How would you describe that?

A: We’re creating a software platform as a way of making Questionmark the perfect complement to other systems people have within their organizations. We are documenting, in an open way, how you can interact with Questionmark software at both the Web service and Web development level to make integration projects go smoothly and easily.

Q: What standards will you be discussing during your Best Practices session?

A:  I am going to be talking about some of the standards in development now that we are are looking at as having the potential for support in the future.  The IMS Global Learning Consortium, for example, are in the process of publishing an interesting specification about how learning tools can be put together – learning tools interoperability.  I will also be talking about standards for exchanging data about people, IMS Learning Information Services and specifications from HR-XML — and the possibilities they open up for synchronizing people and organizational information with Perception. I’m also be going to talk about some of the more generic standards that might be having an impact on us too, such as XHTML 5 and the emerging standards for authentication and authorization such as OpenID and OAuth.  And of course, I’ll be touching on SCORM, too, although I will also be co-facilitating a separate discussion devoted entirely to the future of SCORM with Daniel Rehak from Advanced Distributed Learning.

Q: How do you see people using the information they learn in your session?

A: I hope people get a deeper understanding of what’s happening in some of these developments and take that back into their own organizations in order to figure out the impact the standards will have on them. As well as providing feedback to Questionmark, I’d like them to feel able to talk about their requirements with all their suppliers and understand how they can influence the ongoing standards process. I hope people take a way more than just the acronyms and what they stand for – I’d like them to get some ideas about how they might start conversations within their own organizations to help them prepare for future adoption.

Q:  What are you looking forward to most at the conference?

A: Of course, like every Product Owner, I’m looking forward to meeting all the new and returning customers!  It will be good opportunity to talk in a little more depth with people about integration issues, how they are thinking of using the new version of Perception and what opportunities our Open Assessment Platform presents. I have to add that I’m looking forward to a bit of sunshine as well!

Learn more about the many  conference activities by visiting And register soon to attend the conference, which will take place March 14 – 17 in Miami.

Assessment Standards 101: IMS QTI XML

john_smallPosted by John Kleeman

This is the second of a series of blog posts on assessment standards. Today I’d like to focus on the IMS QTI (Question and Test Interoperability) Specification.

It’s worth mentioning the difference between Specifications and Standards: Specifications are documents that industry bodies have agreed on (like IMS QTI XML), while Standards have been published and committed to by a formal legal body (like AICC or HTML). A Specification is less formal than a Standard but still can be very useful for interoperability.

Questionmark was one of the originators of QTI. When we migrated our assessment platform from Windows to the Web in the 1990s, our customers had to migrate their questions from one platform to the other. As you will know, it takes a lot of time to write high quality questions, and so it’s important to be able to carry them forward independently of technology. We knew that we’d be improving our software over the years and we wanted to ensure the easy transfer of questions from one version to the next. So we came up with QML (Question Markup Language), an open and platform-independent method of maintaining questions that makes it easy for customers to move forward in the future.

Although QML did solve the problem of moving questions between Questionmark versions, we met many customers who had difficulty bringing content created in another vendor’s proprietary format  into Questionmark. We  wanted to help them, and we also wanted to embrace openness and allow Questionmark customers to export out their questions in a standard format if they ever wanted to leave us. So we worked with other vendors within the umbrella of the IMS Global Learning Consortium to come up with QTI XML, a language that describes questions in a technology-neutral way.  I was involved in the work defining IMS QTI as were several of my colleagues: Paul Roberts did a lot of technical design, Eric Shepherd led the IMS working group that made QTI version 1, and Steve Lay (before joining Questionmark) led the version 2 project.

Here is a fragment of QTI XML and you can see that it is a just-about-human-readable way of describing a question.

<?xml version="1.0" standalone="no"?>
<!DOCTYPE questestinterop SYSTEM "ims_qtiasiv1p2.dtd">
<item title="USA" ident="3230731328031646">
<mattext texttype="text/html"><![CDATA[<P>Washington DC is the capital of the USA</P>]]></mattext>
<response_lid ident="1">
<render_choice shuffle="No">
<response_label ident="A">
<material> <mattext texttype="text/html"><![CDATA[True]]></mattext> </material>
<response_label ident="B">
<material> <mattext texttype="text/html"><![CDATA[False]]></mattext> </material>
<outcomes> <decvar/> </outcomes>
<respcondition title="0 True" >
<conditionvar> <varequal respident="1">A</varequal> </conditionvar>
<setvar action="Set">1</setvar> <displayfeedback linkrefid="0 True"/>
<respcondition title="1 False" >
<conditionvar> <varequal respident="1">B</varequal> </conditionvar>
<setvar action="Set">0</setvar> <displayfeedback linkrefid="1 False"/>
<itemfeedback ident="0 True" view="Candidate">
<itemfeedback ident="1 False" view="Candidate">
QTI XML has successfully established itself as a way of exchanging questions. For a long time, it was the most downloaded of all the IMS specifications, and many vendors support it. One problem with the language is that it allows description of a very wide variety of possible questions, not just those that are commonly used, and so it’s quite complex. Another problem is that (partly as it is a Specification, not a Standard) there’s ambiguity and disagreement on some of the finer points. In practice, you can exchange questions using QTI XML, especially multiple choice questions, but you often have to clean them up a bit to deal with different assumptions in different tools. At present, QTI version 1.2 is the reigning version, but IMS are working on an improved QTI version 2, and one day this will probably take over from version 1.