An update on electronic standards for higher education

Posted by Steve Lay

Last month I attended the PESC Spring Data Summit. PESC stands for Postsecondary Electronic Standards Council, a forum in which users and vendors can come together to help align the way data in Higher Education (HE) is collected and exchanged.

The world of electronic standards is a minefield of acronyms and I heard the usual joke about “so many to choose from” at least once during the 3-day meeting. However, each organization tends to represent a specific relationship between a group of users and their technology suppliers. College admission has a similarity to recruitment: applicants are selected and test scores are often involved, but they are carried out by different groups of stakeholders with different data requirements. This is why standards bodies like HR-XML and PESC are both important. There is no single electronic standard that can satisfy both sets of use cases even though tools like Questionmark Perception can be used in both applications.

During the summit, the US Department of Education (DoE) announced the conclusion of their investigation into electronic standards in the assessment domain. They received detailed responses to a Request for Information from a wide range communities and have published a summary of the responses on the website.

The document contains a useful model of the key elements of the assessment process and the standards that help integrate them together:

I. Assessment Instruments and Items: Format and Packaging (Questionmark: Authoring)

II. Initiation and Return of Assessment Administrations (Questionmark: Scheduling)

III. Administration of Assessments (Questionmark: Delivery)

IV. Learning Outcomes Management (Questionmark: Reporting)

V. Learning Records Management

(Customers familiar with the Questionmark wheel logo may see a similarity with our own model of the assessment process!)

The summary will not be surprising to anyone who has worked with the standards bodies. In my opinion it is a fair description of the current state of affairs. It reaffirms my belief that IMS QTI/Common Cartridge used in conjunction with Learning Tools Interoperability (LTI) remains the best route forward to improving integration in the content Authoring and Scheduling elements of the process.

Finally, one area to watch for the future:  At the data summit, PESC announced a strategic partnership with InCommon, the access management federation for US Higher Education. The federation provides a legal and technical framework to help improve access management across this community. It applies across all tools, not just assessment. On reading the DoE recommendations in the Technology Landscape section of the summary, I see a strong resonance with the goals and activities of the federation.

Conference Close-up: Technical Standards and Questionmark’s Open Assessment Platform

Posted by Joan Phaup

The program for the  Questionmark Users Conference is full of informative sessions about everything from item and tests analysis analytics to techniques for writing assessments measure skill and ability as well as knowledge. Questionmark Intetegration Team Lead Steve Lay will conduct a session on Technical Standards and Questionmark’s Open Assessment Platform.

Q: What’s your role at Questionmark?

Steve Lay

A: I’m the Integration Team Lead, so I look after all aspects of the product  concerned with integrating Questionmark with other systems — both within an organization and over the Internet.  I am Product Owner for our Connector products, which enable integrations of Questionmark Perception with other key systems such as Sharepoint Portal Server, Blackboard, Moodle and others. And I am heading our Open Assessment Platform Initiative.

Q: You just mentioned the Open Assessment Platform. How would you describe that?

A: We’re creating a software platform as a way of making Questionmark the perfect complement to other systems people have within their organizations. We are documenting, in an open way, how you can interact with Questionmark software at both the Web service and Web development level to make integration projects go smoothly and easily.

Q: What standards will you be discussing during your Best Practices session?

A:  I am going to be talking about some of the standards in development now that we are are looking at as having the potential for support in the future.  The IMS Global Learning Consortium, for example, are in the process of publishing an interesting specification about how learning tools can be put together – learning tools interoperability.  I will also be talking about standards for exchanging data about people, IMS Learning Information Services and specifications from HR-XML — and the possibilities they open up for synchronizing people and organizational information with Perception. I’m also be going to talk about some of the more generic standards that might be having an impact on us too, such as XHTML 5 and the emerging standards for authentication and authorization such as OpenID and OAuth.  And of course, I’ll be touching on SCORM, too, although I will also be co-facilitating a separate discussion devoted entirely to the future of SCORM with Daniel Rehak from Advanced Distributed Learning.

Q: How do you see people using the information they learn in your session?

A: I hope people get a deeper understanding of what’s happening in some of these developments and take that back into their own organizations in order to figure out the impact the standards will have on them. As well as providing feedback to Questionmark, I’d like them to feel able to talk about their requirements with all their suppliers and understand how they can influence the ongoing standards process. I hope people take a way more than just the acronyms and what they stand for – I’d like them to get some ideas about how they might start conversations within their own organizations to help them prepare for future adoption.

Q:  What are you looking forward to most at the conference?

A: Of course, like every Product Owner, I’m looking forward to meeting all the new and returning customers!  It will be good opportunity to talk in a little more depth with people about integration issues, how they are thinking of using the new version of Perception and what opportunities our Open Assessment Platform presents. I have to add that I’m looking forward to a bit of sunshine as well!

Learn more about the many  conference activities by visiting And register soon to attend the conference, which will take place March 14 – 17 in Miami.

Questionmark re-certified by HR-XML

john_smallPosted by John Kleeman

The HR-XML Consortium is an independent organization that develops standard XML specifications to allow easier communication between human resources-related (“HR”) systems. HR-XML specifications tend to be short and to the point and easy and practical to develop for, so HR-XML has got a lot of traction over the years as a good vehicle for communication between different HR software systems. Questionmark is pleased to be a member of HR-XML.

Happy Birthday to HR-XML! HR-XML is celebrating its 10th anniversary on 2nd October at a luncheon at the HR Technology Conference in Chicago.

Questionmark has supported HR-XML in Questionmark Perception since 2005 and been certified by HR-XML to validate our conformance. Our support consists of providing a web services API within QMWISe that exports a Questionmark assessment result in HR-XML format. This allows other systems that can read HR-XML to easily process assessment results.

The example piece of Perception HR-XML output below shows that Jane Smith has taken a Microsoft Office Skills Test and passed it with a score of 85%. The XML also includes her topic scores, the date she took the test and the Questionmark result ID in case more information is needed. It’s practical, short and can be read by a person as well as by a computer.

<?xml version="1.0" encoding="utf-8"?>
<AssessmentResult xmlns:xsi="" xmlns:xsd="" xmlns="">
<ClientId idOwner="Questionmark">
<IdValue name="LicenseID">41025000000000001437</IdValue>
<IdValue name="Participant">Jane Smith</IdValue>
<IdValue name="Assessment">Microsoft Office Skills Test</IdValue>
<IdValue name="ResultID">20438508</IdValue>
<Score type="percentile">85</Score>
<Comments> </Comments>
<Description>Microsoft Office\Word</Description>
<Score type="percentile">80</Score>
<Description>Microsoft Office\Powerpoint</Description>
<Score type="percentile">80</Score>
<Description>Microsoft Office\Excel</Description>
<Score type="percentile">100</Score>
<Description>Microsoft Office\Outlook</Description>
<Score type="percentile">80</Score>
<UserArea />

HR-XML Consortium certification specialists validate the structure and meaning of sample data in order to provide a certification. Information about certification and a list of certified vendors like Questionmark is available at Certified organizations need to re-certify every two years to ensure software still meets the standard.

I’m pleased to let you know that Questionmark has just completed its re-certification (see here for details). I’d encourage anyone looking for a simple assessment results exchange format to look at HR-XML because it’s simple, practical … and of course Questionmark supports it.

Seven years of web services for easier integrations

john_smallPosted by John Kleeman

A key objective for Questionmark Perception has been to make it an open system that handles integrations easily. Assessment isn’t usually standalone; most organizations need to integrate it with other organizational systems. There are many ways to integrate with Perception, including via our support of standards such as AICC, HR-XML and SCORM, but where standards are not available we recommend integration via our QMWISe web services.

Although web services are routine today, Questionmark adopted them very early: June 6th, 2009, marks the 7th anniversary of Questionmark’s web services, which we call QMWISe. (See our 2002 press release here.)

Two great advantages of web services are that you can call them from almost any platform or system and they are independent of the technology used. So you can code web services in almost any programming language or environment and interface with Questionmark Perception.

Another beauty of web services is that code written back in 2002 will still work in 2009,and code written today should still work in 2016! In the last seven years, there have been very substantial changes to the Questionmark Perception database format and to the user interfaces, but the APIs (Application Programming Interfaces) remain the same. And exactly the same code written then to call QMWISe will still work now. We have ambitious plans to continue developing Questionmark software in new ways, but code our customers write today for QMWISe will still work in the future.

Back in 2002, there were 37 web services methods. Over the years, we’ve added lots more methods and there are now 109. Example web services methods are to create a participant, schedule a participant or give a URL to get access to an assessment.

Many of our customers use QMWISe to integrate with Perception, so that as Perception versions change, their code can remain safe. We or our partners have also used QMWISe to build connectors to many other systems, including Blackboard, Moodle and uPortal. We also call QMWISe within our own software. For instance, Questionmark to Go passes all its results back via web services, and in the future we’ll be trying to use QMWISe more within other code–to “eat our own dog food” and ensure that QMWISe is fully able to be mission critical. By using web services within our own code, we will be driving QMWISe forward to cover more capabilities and so open up the platform to support a wide range of solutions integrated with third party applications.

One key lesson that we’ve learned over time with web services is that commitment and continuity are vital. No one wants to interface with a system that will change. And you need to have good documentation with examples, good scalability and good diagnostics–for instance a log of all SOAP traffic. We recommend that other developers consider making web services available from their own systems: it’s an excellent way of integrating.

In the future we’ll be announcing further improvements to QMWISe that should make it more useful for developers and provide easier ways for customers to integrate with Perception. Questionmark strongly recommends that anyone developing integration into our software uses our web services. We welcome questions, comments and suggestions for improvements, so let us know what you think!