Considering key applications for mobile assessments

Posted by John Kleeman

Questionmark have recently introduced multi-lingual apps to deliver assessments on Android, iPhone and iPad mobile devices, and I spent time at the European Users Conference last week demonstrating our Android app on my phone. In listening to our users talk about how they might deploy assessments on mobile devices, two areas attracted the most interest:

  1. Formative assessments. Assessments help learning; so giving quizzes to people on the device that they carry around with them all the time is interesting to many, and is easy to do. It’s attractive that with Questionmark, you author the assessments once, and participants can take them on their device of choice.
  2. Observational assessments. There is a big need to assess people doing things in work situations, whether it’s how a welder uses a machine, how a nurse interviews a patient or how a soldier cleans his equipment. It’s not practical to carry a laptop into many work environments, but it is practical to fill in questions on a mobile device while observing someone, so this possibility attracted a lot of interest.

Questionmark will shortly be improving our software for use for observational assessments – so watch this space.

P.S. I’m very excited to hear that Bryan Chapman is going to be the keynote speaker for our next users conference in Los Angeles in March 15 – 18, 2011. He is a real thought leader in our industry and someone I admire greatly – he introduced me and Questionmark to the AICC standard back in the 1990s and encouraged us to support it  –  and we’ve now been re-certified 5 times!

Assessment standards 101: AICC

john_smallPosted by John Kleeman

Effective assessment often needs to be integrated with other systems. Some integrations are proprietary, but wherever possible Questionmark tries to integrate using technology standards, as these are longer lasting than proprietary solutions and allow us to build one solution which can work for many customers.

Over the years Questionmark has been involved in many standards initiatives, and I thought I’d share in a series of blog articles a personal perspective of some of the key standards that impact assessment.
I’ll start with the standard commonly called AICC or AICC HACP (more formally AICC AGR-10), which is used by learning management systems to call assessment content. Millions of Questionmark assessments are called each year via AICC, and it’s the most successful of all the standards we use.

The AICC is an aviation industry organization founded in 1988, the same year Questionmark. Airlines and airplane makers wanted a way to deliver computerized learning to help people maintain planes that could last the 20 years or so years that the planes themselves would last. The original AICC standard was file based but was soon updated to work over HTTP.

In an assessment context, the AICC standard allows launching and tracking of an assessment:

1.  A Calling Application (for instance an LMS) calls an assessment system saying that it wants to start an assessment.

2.  The assessment system asks the Calling Application for the details of which participant and which assessment.

3.  The Calling Application replies and the assessment starts.

4.  At the end of the assessment, the score is passed back to the Calling Application for tracking.

A key reason the AICC standard is robust and successful is that there is direct server-to-server communication. The two pieces of software communicate directly by HTTPS, and so there is no possibility of disruption or interference by anything at the participant workstation.


I was introduced to the AICC standard in the 90s by Bryan Chapman (now an e-learning analyst) and Questionmark supported it as a way of making it easy for people with LMSs to call Questionmark Perception assessments. We first became certified to the AICC standard in 1999 and we’ve been re-certified several times since. The great thing about the AICC standard is that it really works: Because it’s been tried and trusted over many years, I can think of at least 25 different vendors that Questionmark has interoperated with using this standard, ranging from PeopleSoft and Sun down to much smaller vendors.