Seven years of web services for easier integrations

john_smallPosted by John Kleeman

A key objective for Questionmark Perception has been to make it an open system that handles integrations easily. Assessment isn’t usually standalone; most organizations need to integrate it with other organizational systems. There are many ways to integrate with Perception, including via our support of standards such as AICC, HR-XML and SCORM, but where standards are not available we recommend integration via our QMWISe web services.

Although web services are routine today, Questionmark adopted them very early: June 6th, 2009, marks the 7th anniversary of Questionmark’s web services, which we call QMWISe. (See our 2002 press release here.)

Two great advantages of web services are that you can call them from almost any platform or system and they are independent of the technology used. So you can code web services in almost any programming language or environment and interface with Questionmark Perception.

Another beauty of web services is that code written back in 2002 will still work in 2009,and code written today should still work in 2016! In the last seven years, there have been very substantial changes to the Questionmark Perception database format and to the user interfaces, but the APIs (Application Programming Interfaces) remain the same. And exactly the same code written then to call QMWISe will still work now. We have ambitious plans to continue developing Questionmark software in new ways, but code our customers write today for QMWISe will still work in the future.

Back in 2002, there were 37 web services methods. Over the years, we’ve added lots more methods and there are now 109. Example web services methods are to create a participant, schedule a participant or give a URL to get access to an assessment.

Many of our customers use QMWISe to integrate with Perception, so that as Perception versions change, their code can remain safe. We or our partners have also used QMWISe to build connectors to many other systems, including Blackboard, Moodle and uPortal. We also call QMWISe within our own software. For instance, Questionmark to Go passes all its results back via web services, and in the future we’ll be trying to use QMWISe more within other code–to “eat our own dog food” and ensure that QMWISe is fully able to be mission critical. By using web services within our own code, we will be driving QMWISe forward to cover more capabilities and so open up the platform to support a wide range of solutions integrated with third party applications.

One key lesson that we’ve learned over time with web services is that commitment and continuity are vital. No one wants to interface with a system that will change. And you need to have good documentation with examples, good scalability and good diagnostics–for instance a log of all SOAP traffic. We recommend that other developers consider making web services available from their own systems: it’s an excellent way of integrating.

In the future we’ll be announcing further improvements to QMWISe that should make it more useful for developers and provide easier ways for customers to integrate with Perception. Questionmark strongly recommends that anyone developing integration into our software uses our web services. We welcome questions, comments and suggestions for improvements, so let us know what you think!

Questionmark Conference: Progress in Criterion Referenced Measurement


Sharon Shrock

joan-small2Posted by Joan Phaup

A highlight of the Questionmark Users Conference in Memphis  was Sharon Shrock and Bill Coscarelli’s keynote, “Results You Can Rely On: What We’ve Learned from 25 Years of Criterion-Referenced Measurement.”

Sharon and Bill took us from the early history of testing to the work of Robert Glaser, who in laying the foundations of  criterion reference testing focused on the importance of setting objectives and measuring test takers against a standard instead of against one another. They had us try William Angoff’s method of setting cut scores ( click here to access a detailed paper on this subject by Bill and Sharon together with Andrew Barrett and John Kleeman) and reviewed Donald Kirkpatrick’s four levels  (“Don’t skip Kirkpatrick Level 2 if you are using 3 and 4!”) .  They also described the

Bill Coscarelli

Bill Coscarelli

six levels of what they call the “Certification Suite” (noted by Dr. Will Thalheimer in his review of  the latest edition of  Shrock and Coscarelli’s book, Criterion-Referenced Test Development.)

A key point was that people on a job are doing much more than remembering facts. So test questions, rather than just operating at the  memory level, should deal with real problems that people who do a particular job have to think through.

Tuesday’s conference schedule also included sessions on how to organize item banks, effective reporting techniques, teaching faculty to use Questionmark Perception, item analysis, and test maintenance best practices.

Wednesday brought some encore tech training sessions and a “road ahead” session seeking participants’ reactions to ideas for future features and services. Then some quick good-byes as participants headed for home. Here’s looking forward to next year’s conference!

Some New Questionmark Web Seminars

joan-small1Posted by Joan Phaup

We have some new web seminars coming up for those of you who want to go into more detail on particular aspects of using Questionmark Perception.

From Item Banking to Content Harvesting: Authoring in Questionmark


Perception — May 7th at 3 p.m. EDT

  • This webinar will demonstrate the use of different authoring tools to author questions for use in surveys, quizzes, tests and exams. Questions from the various sources will be assembled into a sample assessments, which will then be taken online. Figure out which tools are the most practical for you and how to make them work together to produce assessments quickly and easily.

Analyzing and Sharing Assessment Results with Questionmark Enterprise Reporter — May 20th at 3 p.m. EDT

  • This session explains each of Perception’s 12 standard reports and the data and statistics they contain. Join us to learn how use templates to help you create reports easily and to learn about the various filter options you can use.

We have two seminars scheduled for Thursday, April 16th:

Overview of New Features in Perception v4.4 , set for 11 a.m. EDT in the U.S.

  • Includes several live demonstrations and gives you the opportunity to ask questions about them.

Beyond Multiple Choice: Nine Ways to Leverage Technology for Better Assessments at 10 a.m. BST in the U.K.

  • Will explore the role of assessments in measuring people’s knowledge, skills and attitudes. Join us to learn techniques for creating effective assessments that will help improve performance, manage workforce competencies, and ensure regulatory compliance.

We continue with a full schedule of introductory webinars for beginners. You can learn more and register for the webinar of your choice  at the following links:

US Webinar Schedule

UK Webinar Schedule