Join us in London or Edinburgh for briefings on assessment security

Chloe MendoncaPosted by Chloe Mendonca

This June, we’re getting together with online invigilation leader ProctorU to deliver breakfast briefings in two UK cities.Proctor U

The briefings, to be held in London on 17th June and in Edinburgh on 18th June, will focus on innovative technologies that make it possible to deliver high-stakes tests using almost any webcam and computer, anywhere in the world .

Online courses help organisations increase accessibility to their programs, but until recently, when it came time for an exam, students had to travel to a test centre. Now, if you can study remotely, it’s equally feasible to you take exams remotely, too. Using online invigilators or proctors is a practical solution for institutions and organisations worldwide — a means of providing secure testing at a distance.

The sessions – co-presented by Questionmark and ProctorU, will explain the basics of online invigilation, discuss proven strategies for alleviating the testing centre burden and explore how the “last mile” of high-stakes test delivery can meet the goals and needs of all stakeholders.bb3

The breakfast briefings will include a complimentary breakfast at 8.45 a.m. followed by presentations and discussions until about 12:30 p.m.

These gatherings are ideal for educators, instructional designers and managers from academic institutions, businesses and other organisations.

The sessions offer an excellent way to learn about the newest online assessment technologies and services. They’re also a great opportunity to meet other assessment professionals in your area.

If you’re new to online assessment or online invigilation, this is an opportunity you don’t want to miss!

Secure Testing in Remote Environments: A SlideShare Presentation

Headshot JuliePosted by Julie Delazyn

How can you be sure that someone taking an online exam away from a testing center or classroom is adhering to the guidelines put in place by your instructional staff?

This SlideShare presentation will demonstrate how instructors can prevent or catch cheating and ensure a secure environment for employees or students taking tests in their homes, offices and other locations.

The slides are from a Best Practices sessions at the 2013 Questionmark Users Conference: Don Kassner of ProctorU discussed strategies for reducing incidents of dishonesty online, and Maureen Woodruff of Thomas Edison State College explained how online proctoring enables the college to administer tests securely to thousands of online learners.

This presentation offers a glimpse into the kind of discussions and sessions you can find at our Users Conferences. Registration is already open for the 2014 Users Conference March 4 – 7 at the Grand Hyatt on the beautiful Riverwalk in San Antonio, Texas. Discounts are available for groups and early registrants. Sign up soon and plan to be there!

Secure exams outside the testing center

Joan Phaup HeadshotPosted by Joan Phaup

The increasing numbers of students studying online in recent years – many of them raising families and holding down jobs – have embraced the idea of doing all their coursework at the kitchen table, so to speak.  But until recently, when it came time for a test, these students had to travel to at testing center. Many of these students raised the question: “If I can study at the kitchen table, why can’t I take an exam there, too?”

Today, taking tests from home or the office — using online monitors or proctors — is an option for certification candidates as well as students, and there are various means of providing secure testing at a distance.

Delegates to the Questionmark Users Conference in Baltimore March 3 – 6 will have the opportunity learn more about online proctoring/invigilation during a presentation on Secure Testing in Remote Environments.

Don Kassner, president of ProctorU, will co-present this session with Maureen Woodruff, who directs the Office of Test Administration at Thomas Edison State College. I spent a few minutes with Don the other day and asked for some details.

Can you explain what makes it possible to offer secure remote proctoring or monitoring?

Don Kassner

Don Kassner

There are three key elements to this: the environment, the computer and the test taker. The first thing we need to do is to make sure each test taker has reliable internet access and is in a fairly controlled environment. This is not about testing anywhere. It’s about testing in an environment that’s predictable. The person gets to choose the place, but it has to be in a certain kind of place. And the test taker must “show” us their environment using a webcam. Second, we have to secure their computer. Test takers use their own equipment, but we need to make sure they are not switching tasks, accessing the Internet for answers and so forth. Last, we have to secure the test taker themselves, by using a layered authentication approach to make sure they are who they say they are having our online proctors observe them as they complete their tests.

What are the biggest security challenges in delivering tests to people outside of test centers?

In a test center, you already know that the environment and the computers are secure; you can focus on the identity and behavior of the test taker.  When proctoring at a distance, you have to give a lot of importance to all three of the elements I mention; there are a lot more decisions to be made about the testing process.

With online proctoring, we have to be willing to stop a testing session and say that something doesn’t meet our standards – that the test taker is not meeting the requirements and must reschedule.

We also need to be able replicate our processes across the board and make sure that the testing experience replicates no matter who is taking the test or where they are taking it. We have to focus on making sure the experience is identical for every test taker.

How will you address those challenges during your session?

We will introduce the basics of online proctoring and give examples of how different institutions and organizations have used it. We’ll also drill down into the details of what it takes to secure the environment, the computer and the test taker.

Woodruff Maureen Exc  Portrait (cropped)

Maureen Woodruff

Maureen will share a case study about what they did at Thomas Edison State College and the important factors they had to take into account when they set up remote testing for their students. And I’ll differentiate between the factors that are important for academic institutions and those that matter the most for certification tests. Students are likely to take a number of tests and end up have a long track record. Certification candidates tend not to be repeat test takers, so that means using slightly different procedures.

What kinds of tests are best suited for online monitoring or proctoring?

If you are going to use this kind of proctoring, you really need to think about the nature and structure of the test. You are trying to minimize the risk inherent in someone taking a test, so you need to ask yourself what issues you are concerned about relative to that. Tests with large data banks are best, because they help mitigate the risks of people stealing questions or colluding. Standard tests  increase the risk factor and may not be appropriate.

What would you like your audience to take away?

A real understanding on how effective this approach can be in some situations and  an understanding of when it may or may not be appropriate – so they can think about their own programs and consider where they think this will fit.

Click here for more information about the conference program — and register soon!



Content protection and secure delivery: Test Design and Delivery Part 9

Posted By Doug Peterson

Writing good items and putting together valid and reliable assessments can take a lot of time and cost a lot of money. Part of an assessment’s reliability and validity is based on the test-taker not knowing the items ahead of time. For these reasons, it is critical that item exposure be controlled.

This starts during the development process by requiring everyone who develops items or assessments to sign a confidentiality agreement. Developers’ computers should, at the very least, be password-protected, and you should consider data encryption as well.

Care must be taken to prevent question theft once an assessment is assembled and delivered. Do not allow overly generous time limits, which would provide time for a test-taker to go back through the assessment and memorize questions. If your assessment is delivered electronically, consider not allowing backward movement through the test. Be very careful about allowing the use of a “scribble sheet”, as someone might try to write down questions and sneak them out of the test center: be sure to number all scribble sheets and collect them at the end of the assessment.

Computer-based testing makes it very easy to utilize random item selection when the assessment is delivered. While this does mean having to develop more items, it cuts down the number of times any one item is delivered and helps to reduce cheating by presenting different questions in a different order to teach test-taker.

It is critical to track the number of times an item has been delivered. After a certain number of deliveries, you will want to retire an item and replace it with a new item. The main factor that impacts how many times an item should be exposed is whether the assessment is high-stakes or low-stakes. Items on a high-stakes exam should have a lower maximum number of exposures, but items on a low-stakes exam can have a higher number of exposures.

As long as there have been tests, there have been test-takers who try to cheat. Make sure that you authenticate each examinee to ensure that the person who is supposed to be taking the exam is, in fact, the person taking the exam. Testing centers typically prohibit talking, using notes, and using cell phones during tests. Maintain a minimum amount of space between test-takers, or use carrels to physically separate them.

Test administrators should walk around the room during the test. Unauthorized personnel should not be permitted to enter the room during the test, and the administrator should not leave the room for any reason without first bringing in another administrator.

Computer-based testing presents its own set of security challenges, especially when testing is permitted outside of a secure testing center (e.g., in the test-taker’s home). Questionmark offers the Questionmark Secure client, which locks  down test-takers’ machines and doesn’t allow them to copy questions or switch tasks.

Computer-based testing security can/should also include some form of identification and password verification. Additionally, in the last few years, technology has become available that allows for the remote monitoring of test-takers using built-in laptop/tablet cameras or small desktop devices.

Click here for links to a complete listing of posts in this series.

October Webinars on remote test monitoring, mobile assessments and more

Joan PhaupPosted by Joan Phaup

With the school year now well underway, we are happy to offer some of our own learning opportunities: three Web seminars on important topics for people involved in assessment and measurement.

You can sign up for any or all of these one-hour sessions free of charge at

Here’s the line-up for October:

Thursday, October 18, at 1 p.m. Eastern Time — Integrity Anywhere: Secure Monitoring of Higher-stakes Online Tests 

Innovative technologies make it possible to take high-stakes tests using almost any webcam and computer, anywhere in the world. Online testing helps organizations increase access to their programs and assessments, but they must balance flexibility, candidate experience, logistics and security in order to ensure the integrity and value of the exams they deliver from a distance. This web seminar, co-presented by Questionmark and ProctorU,  explores the “last mile” of high-stakes test delivery and how it can meet the goals and needs of all stakeholders.

Tuesday, October 30, at 1 p.m. Eastern Time —  Creating Assessments for Mobile Delivery

Many learning professionals are considering the use of popular and inexpensive mobile devices such as smart phones and/or tablets for delivering assessment content. This seminar will show you how to use mobile assessment delivery  on its own or as part of a blended delivery strategy to give learners retrieval practice, gather their opinions or test their knowledge while they are on the move.

October 12, 24 and 31 at assorted times: Introduction to Questionmark’s Assessment Management System

This Webinar is for people who are new to Questionmark and want an overview of our assessment management technologies. A Questionmark expert will walk you through the basics of authoring, delivering and reporting on surveys, quizzes, tests and exams. It explains the key features and functions available in Questionmark OnDemand and Questionmark Perception.


Delivering tests to Mac users via Questionmark Secure

Posted by Jim Farrell

It’s no secret that Macs are everywhere. Visit a college campus and you see the distinctive logo on laptops as far as the eye can see. We are pleased to announce a new option for customers wishing to deliver exams on a locked-down browser: Questionmark Secure is now available for Mac users.

For those of you who don’t already know, Questionmark Secure is a lock-down browser that people can use to take online tests and exams. Organizations can use it in conjunction with Questionmark OnDemand or Questionmark Perception on-premise software to help test takers use their own cognition to answer questions and prevent them from accessing other applications.

Questionmark Secure was already available for Windows PCs, and we’re pleased to be extending secure delivery across a broader range of platforms with the addition of this new version for Macs. It’s important to be able to deliver tests securely regardless of whether participants are using Windows or Mac machines.

Click here for details about Questionmark Secure for Windows and Mac PCs, including links to free downloads.