What’s the key to reliable assessment management?

Posted by John Kleeman

If you are choosing a platform for online assessments, what is your most important criterion? Probably that the system is reliable and trustable. An online assessment is only useful if participants, instructors and other stakeholders trust the results.

But if you are building an assessment system, how do you make sure it is reliable? How do you ensure scores are added up correctly, results don’t get lost, reports are accurate and that your system doesn’t get broken when a new browser comes out?

One of the things we’ve learned at Questionmark is that using a three-tier architecture helps hugely with QA and reliability. In much of the Questionmark software, there are three tiers, which we regard as key ingredients for trustable assessment management.

  • The Presentation tier deals with the user interface. For example, it formats assessments for display
  • The Business tier contains the business logic. For example, the scoring of an assessment
  • The Data tier records results and other data in a robust database

(Questionmark also has a Services Layer between the Presentation and Business tiers, which allows the business logic to be called from other applications and for testing purposes.)

Questionmark 3 tier architecture, presentation tier, business tier, data tier and services layer

So why does this matter? Why should you care if your assessment application has three tiers? Here are four reasons:

1. It gives a way to test the logic in the Business tier independently. So you can set up a range of automated tests with a variety of input/output at the business tier and test this thoroughly, independently of the user interface. If you don’t automate testing, then mistakes will creep in; if you automate at the UI level, you have to change the test scripts whenever browsers change.

2. You need to be able to update the Presentation tier frequently to take account of changes in browsers, new mobile devices and so on, without having to also change and risk errors in the Business tier.

3. Three tiers make an important difference to security. The Presentation tier protects the other tiers from inappropriate calls, and you can put firewalls between each tier, to protect the data and integrity of the system.

4. A tiered system is much easier to load-balance and scale. You can assign the right number of servers to each tier, and when you get a bottleneck, increase the number of servers in that tier.

So if you’re looking for a platform to run online assessments on, it’s worth asking your supplier:

  • Do you have a three-tier architecture?
  • Do you do automated service testing under the user interface layer?

If the answers to these questions are “no”, you might want to ask how they can be sure their system will stay reliable as they update it.


Sharing test results automatically via email

Posted by Joan Phaup

When someone passes a test, it’s easy to tell them the good news automatically via email.

To do so, you need to have a Simple Mail Transfer Protocol (SMTP) service running on the application server. Then set up your test to associate an automatic email with a particular assessment outcome.

When you open Questionmark’s Authoring Manager and locate your assessment in your repository, use the Assessment Editor to enable “Email” from a list of Assessment Outcome settings and tick “Participant” as the email recipient. The
email will be sent when the participant achieves the passing score.

This is just one type of email you can request. You could also use emails to tell managers how people who report to them have performed on a test – or to inform instructors of people who need remedial instruction. And you are not limited to pass-fail notifications: you can include details about scores and other details, too.

LTI and the future of learning management systems

Jeff Place at the Questionmark booth at The Blackboard Developer Conference

Posted by Steve Lay

Following along with the twitter feed at #devcon12 (The Blackboard Developer Conference) this week, I noticed an interesting theme developing. Here’s one example, from @certtechpro

More LMS’s in the future, but using the same tools? Will LMS of the future be specialized? #devcon12 #bbw12

And another, from @rossmackenzie

Interest theme developing about specialised ( e.g. Subject specific) Learning management systems. #devcon12 #bbw12

These tweets happened during a keynote panel session that grappled with the role of the LMS and the effect that adoption of the IMS LTI standard will have.

I had a follow-up conversation with two members of the panel afterwards to help get some clarification on their vision. In particular, I wanted to reconcile this multiplicity of LMSs with another theme discussed during the session: the importance of ensuring that the LMS does not just aggregate new features but “hardens” around core functions, with additional features being incorporated by linking to other tools using IMS LTI.

Using the LTI standard, the LMS can aggregate best-of-breed tools rather than having to develop them specifically as features of the LMS itself.

There seem to be two possible outcomes here:

(1) By being smaller, the LMS could be customised to target specific education or training sectors; it could even be optimised for subject-specific learning methods. The number of LMSs in the market would increase.

(2) Alternatively, by being smaller the LMS could fade into the background, with student interaction being concentrated in the tools that the LMS aggregates. The community is likely to organise itself around a smaller number of more robust systems.

In either case, integration using the LTI standard will make it easier for instructors and programme managers to adopt best-of-breed tools like our own Assessment Management System without having to deploy an LMS-specific connector!

Partnering with ProctorU to Add Security to Higher Stakes Online Exams

Posted by Jim Farrell

On June 7th, Questionmark and ProctorU announced their partnership to add security to higher stakes exams taken outside a test center. For those of us within Questionmark, this was a very exciting announcement as it provided our customers with options that until recently were very limited.

For those of you unaware of ProctorU, let me give you some background. In the simplest of terms, ProctorU is an online, real-time proctoring service. Using webcams, screen-sharing technology and proven authentication techniques, live monitors oversee participants taking their examinations from home, work or anywhere. The monitor can see and hear the examinee and provide pre-exam assistance and technical support, maintaining the integrity of the testing process. ProctorU brings the test center to the test taker.

At Questionmark, we see Impersonation, Content Theft, and Cheating as serious threats to test security. Real-time monitoring directly combats each of these threats by:

  • Preventing ID Fraud – Each monitoring session starts with authentication techniques that ensure the test taker is who they say they are.
  • Protecting Content – ProctorU sessions are monitored in real time with screen sharing. If a monitor detects an anomaly, he/she can immediately end the session and send the video to someone for review before your content hits the Internet.
  • Minimizing Cheating – As part of the pre-test checking, the participant must scan the room and work area with his or her webcam. The monitor then watches the room via the webcam, listens via the computer’s microphone and uses screen sharing to monitor the test taker’s actions.

This is exciting to us because Questionmark’s goal is to provide the appropriate amount of security for the situation the test requires. Is this technology necessary for all types of assessments? No, of course not. But as more people work remotely and more students take course and tests online, there are plenty of assessments that require monitoring. Questionmark’s relationship with ProctorU allows for an easy-to-deploy, seamless, secure and fair experience for the test taker.

If you would like to know more about how live online proctoring works, you can tune in to a podcast with Jarrod Morgan of ProctorU.

Easier collaboration in Questionmark Live

Posted by Julie Delazyn

We always like sharing news about Questionmark Live, our browser-based assessment authoring tool, on the blog, including the recent addition of hierarchical topics. Another big change is the new interface, which makes it easier than ever for subject matter experts and test designers to collaborate on an assessment. We’ve made it very easy to share topics and subtopics.

How does it work? Simply click the share button, as you can see in the screen shot below, and type the email of the person you would like to share the subtopic with. The recipient will receive an invitation to view the topic in Questionmark Live (to see context) and the sub-topic they can work in. It’s that easy.

The subtopic folder that you shared will now display a green arrow, as shown in the screenshot below. Click on the number beneath the “No. of Revisions” column to track a question’s revision history as well as to see who edited the question and which changes were made. You can also compare the different versions of the same question and roll back to a previous version. Learn more about revision history in Questionmark Live in this blog post.

P.S. An “encore” webinar on using assessments for financial services compliance

Posted by Chloe Mendonca

Last week I announced a webinar in the UK on Using Assessments to Mitigate Risk and Ensure Regulatory Compliance.

This subject is so timely that we have added an “encore” to our schedule. You can catch this webinar again at 11 a.m. British Summer Time (London — UTC +1) on Tuesday, 17 July.

Much of the seminar will draw on our white paper, The Role of Assessments in Mitigating Risk for Financial Services Organizations.

Click here to download the paper and here to sign up for the webinar.