Infographic: Online or Test-Centre Proctoring?

Julie ProfilePosted by Julie Delazyn

For many exams, candidates are required to travel to brick-and-mortar test centers where proctors (or invigilators) supervise the process; However, a new way of proctoring certification exams is rapidly gaining traction. Two of the world’s largest software companies, SAP and Microsoft, offer online proctoring for their certification programs, and many other companies are looking to follow suit.

Do you need to understand the key differences and benefits? Here’s an infographic that explains some of the pros and cons of the two approaches.

Proctoring Infographic

For more on online proctoring, check out this informational page and video below:

 

 

New best practice webinars: Taking your assessments from to good to great

Posted by Chloe Mendonca

“Good, better, best. Never let it rest. ‘Til your good is better and your better is best.” This old little rhyme teaches us a valuable lesson: There is always room for improvement! No matter what role or business you’re in, if you’re interested in long-term success, you should strive to continuously improve your knowledge, systems and processes.

But how does this relate to assessments? Well, in many ways, there are always things we can do to develop better assessments: more secure, more trustworthy assessment programs. Maybe your current assessment program is “good”, but is “good” good enough?

We’re offering two new webinars that will help you assess how you’re currently performing in two key areas — and take your assessments from good to great:

  1. Item Writing

How to write high quality test items [35-Minute Session]

  • 3rd August, 2016, 3:00 p.m. UK BST / 10:00 a.m. US EDT

Are your items poorly written? Perhaps they’re good but you want them to be “better”. Skilfully crafted items promote learning and memory recall. They help retain knowledge, skills and/or abilities over time, but writing high-quality items isn’t as easy as it looks. This session will give you tips for taking your items to the next level.

  1. Exam Integrity

Enhancing exam integrity with online proctoring [45-Minute Session]

  • 9th August, 2016, 3:00 p.m. UK BST / 10:00 a.m. US EDT

With online proctoring rapidly gaining the attention of organisations and test sponsors around the world, many are wondering how it compares with traditional test centre proctoring. This 45-minute webinar will discuss what online proctoring is, how it works and whether it can in fact boost test security. Don’t miss this session if you’re keen to extend geographic reach and lower test administration costs.


If you’re looking to learn more about what you can achieve with Questionmark’s Assessment Management System, join our 60-minute introductory session. We’ll demo the platform live and cover a number of key features and functions. Save your seat at one of these sessions:

Intro to Questionmark’s Assessment Management System [60-Minute Session]

  • 4th August, 2016, 10:30 a.m. (BST) UK
  • 10th August, 2016, 12:00 p.m. (EDT) US

We also deliver this webinar in Spanish and Portuguese. Check out the upcoming dates and times here.

Online Proctoring: FAQs

John Kleeman HeadshotPosted by John Kleeman

Online proctoring was a hot-button topic at Questionmark’s annual Users Conference. And though we’ve discussed the pros and cons in this blog and even offered an infographic highlighting online versus test-center proctoring, many interesting questions arose during the Ensuring Exam Integrity with Online Proctoring  session I presented with Steve Lay at Questionmark Conference 2016.

I’ve compiled a few of those questions and offered answers to them. For context and additional information, make sure to check out a shortened version of our presentation. If you have any questions you’d like to add to the list, comment below!

What control does the online proctor have on the exam?

With Questionmark solutions, the online proctor can:

  • Converse with the participant
  • Pause and resume the exam
  • Give extra time if needed
  • Terminate the exam

What does an online proctor do if he/she suspects cheating?

Usually the proctor will terminate the exam and file a report to the exam sponsor.

What happens if the exam is interrupted, e.g. by someone coming in to the room?

This depends on your security protocols. Some organizations may decide  to terminate the exam and require another attempt. In some cases, if it seems an honest mistake, the organization may decide that the proctor can use discretion to permit the exam to continue.

Which is more secure, online or face-to-face proctoring?online proctoring

On balance, they are about equally secure.

Unfortunately there has been a lot of corruption with face-to-face proctoring, and online proctoring makes it much harder for participant and proctor to collude as there is no direct contact, and all communication can be logged.

But if the proctors are honest, it is easier to detect cheating aids in a face-to-face environment than via a video link.

What kind of exams is online proctoring good for?

Online proctoring works well for exams where:

  • The stakes are high and so you need the security of a proctor
  • Participants are in many different places, making travel to test centers costly
  • Participants are computer literate – have and know how to use their own PCs
  • Exams take 2-3 hours or less

If your technology or subject area changes frequently, then online proctoring is particularly good because you can easily give more frequent exams, without requiring candidates to travel.

What kind of exams is online proctoring less good for?

Online proctoring is less appropriate for exams where:

  • Exams are long and participants needs breaks
  • Exams where participants are local and it’s easy to get them into one place to take the exam
  • Participants do not have access to their own PC and/or are not computer literate

How do you prepare for online proctoring?

Here are some preparation tasks:

  • Brief and communicate with your participants about online proctoring
  • Define clearly the computer requirements for participants
  • Agree what happens in the event of incidents – e.g. suspected cheating, exam interruptions
  • Agree what ID is acceptable for participants and whether ID information is going to be stored
  • Make a candidate agreement or honor code which sets out what you expect from people to encourage them to take the exam fairly

I hope these Q&A and the linked presentation are interesting. You can find out more about Questionmark’s online proctoring solution here.

Online Proctoring – An Invasion of Privacy?

Steve Lay Headshot

Posted by Steve Lay

Many organisations looking to expand their online offerings now use a new method to securely deliver high-stakes exams online: Online proctoring. A live proctor uses your computer’s webcam to observe you taking the test, to ensure its integrity. To make sure you work alone, the proctor asks you to scan your webcam around the room you are in. The proctors also asks you to show photo ID to verify your identity and will use screen-sharing technology to view your computer screen. In addition, secure browser software can sometimes be used to restrict other computer applications (such as opening a web browser) to restrict a test-taker from accessing digital resources.

Being watched in this way during an online exam often poses questions about privacy…

Is online proctoring an invasion of privacy? Do proctors still have access to your computer after the exam is complete? What sort of things can they access while you’re taking the exam? Can they access your files and identifiable information?

A video link with an online proctor invades no more privacy than taking an exam at a traditional face-to-face test centre. In many cases, allowing a proctor to see everything on your computer screen is just like a proctor at a test centre who can look over your shoulder, see your computer screen and prevent any restricted behavior.  But some online proctoring systems go even further, providing proctors with full control over a candidate’s computer.

Having a proctoring service take control of a candidate’s computer can often be quite helpful.  For instance a proctor who is trained in diagnosing and correcting setup issues can help speed up a process and can quickly resolve problems with the video or audio on the computer. A proctor can also guide the candidate through the exam software, in some cases entering special purpose access credentials that have not previously been made available to the candidate.

Although screen sharing and remote control solutions can be used with Questionmark Online Proctoring, there are alternatives for situations in which such far-reaching access to the candidate’s laptop is inappropriate. Using Questionmark Secure in conjunction with Questionmark OnDemand supports a special mode for online proctoring that gives the proctor limited proxy controls instead of complete control over the machine. For example the proctor can manage the running of the assessment without having control over the participant’s machine. The sense of ‘control’ that many proctoring solutions require here is  similar to popular screen sharing systems that allow you to “Give Control” or “Request Control”. Questionmark Online Proctoring does not require this, because the proctor is connected directly to Questionmark’s service and can manage the exam without going ‘through’ the participant’s computer.

In addition to the privacy advantages of these proxy controls for the candidate, this arrangement also enables the test content to be kept hidden from the proctor. This could provide advantages to the test provider over and above what can be achieved even in a test centre.  The proxy controls allow the proctor to pause the test, add extra time and even terminate the test completely. Meanwhile, Questionmark Secure takes care of monitoring the local computer for signs of misuse and flagging or preventing attempts to cheat.  Questionmark Secure can be audited and installed by a trusted system administrator for a company-owned laptop without having to provide the same permissions to the end user.  Questionmark Secure does not install keylogging software, or any other persistently active service.  It is only active during the exam process itself.

Interested in learning more about Online Proctoring? I will be presenting a session on ensuring exam integrity with online proctoring at Questionmark Conference 2016: Shaping the Future of Assessment in Miami, April 12-15.

There’s only one day left to take advantage our earl-bird savings…click here to register and learn more about this important learning event. See you in Miami!

Certification in the Cloud and the Move to Online Proctoring: An interview with SAP’s manager of global certification

John Kleeman Headshot

Posted by John Kleeman

I recently interviewed Ralf Kirchgaessner, SAP’s manager of global certification, about how the cloud is changing SAP certification. This is a shortened version of my conversation with Ralf. To read the full previously published post, check out this SAP blog.

John: What are the key reasons why SAP has a certification program?

Ralf: The overall mission of the program is that every SAP solution should be implemented and supported ideally by a certified SAP resource. This is to ensure that implementation projects go well for customers, and to increase customer productivity while reducing their operating costs. Customers value certification. In a survey of SAP User Group customers in Germany and the US, 80 percent responded that it was very important to have their employees certified and over 60 percent responded that certification was one of the criteria used to select external consultants for implementation projects.

John: What important trends do you see in high tech and IT certification?

Ralf: What comes first to the mind is the move to the cloud. Throughout the technology industry, the cloud drives flexibility and making everything available on demand. One aspect of this is that release cycles are getting quicker and quicker.

For certification, this means that consultants and others have to show that they are always up to date and are certified on the latest release. It’s not enough to become certified once in your lifetime: you have to continually learn and stay up to date. But of course if you are taking certification exams more often, certification costs have to be much lower. In some regions, people have to travel large distances to get to a test centre. With more frequent certification, it’s not practical to travel to a testing centre every time you take a certification. So our aim is to allow certification anytime and anywhere using the cloud.

John: How does online proctoring work for the candidate?

Ralf: A remote proctor monitors the candidate via a webcam, and there are a lot of security checks done by the proctor and by the system. For example, a secure browser is used, the candidate has to do a 360 degree check of his or her room, and there are lots of specific controls. For instance, you aren’t allowed to read the questions silently with your lips in case someone is watching or listening.

The great advantage to the candidate is flexibility. If someone says, “I’d like to do my exam in the middle of the night or on weekends because during the week I’m so busy with my project,” they can. They might say that they’d like to do their exam on Saturday afternoon: “After spending two hours playing with my kids, I’m relaxed to do my exam!” It’s such a flexible way to get certified and to quickly demonstrate that they have up-to-date knowledge and are allowed to provision customer systems.

John: Who benefits from certification in the cloud? Candidates, customers, partners or SAP?

Ralf: Of course, I think all benefit! Candidates have flexibility and lower cost. Customers can be sure that partner consultants who work for them are enabled and up to date. For partners, it’s a competitive advantage to show that their consultants are up to date, especially for new technologies like S/4HANA and Simple Finance. A partner is much more likely to be chosen to deploy new technologies if they can demonstrate that they have several consultants already certified in something that’s just been released. And for SAP, our goal is to have engaged consultants, happy partners and lower support costs. So everyone genuinely benefits.

John: What are some of the challenges?

Ralf: One example is that it’s important in cloud certification to get data protection right. SAP have very detailed requirements that we ensure our vendors like Questionmark meet.

Security is also a challenge. You need to prevent cheating and stealing questions.  And interfaces and integration need to be right. We have worked out how we get the data from our HR systems, how people book and subscribe to exams and then how they can authenticate with single sign-on into the certification hub to take cloud exams.

The delta concept also gives challenges. You need very precise pre-requisite management logic, where the certification software checks for example that, if you want to take the delta exam, you have already passed the core exam. It also can sometimes be difficult to prepare a good delta exam, particularly if a new release has very specific or detailed features, including some that apply in only some industries.

Lastly, providing seamless support is a challenge when using multiple vendors. The candidate doesn’t care where a problem happened: he or she just wants it fixed.

John: Where do you see the long term future of high-tech certification? Will there still be test centres, or will all certification be done via the cloud?

Ralf: Test centres won’t disappear at once, but there is a trend of moving from classroom-based learning and testing to learning and certification in the cloud. The future will belong to anytime, anywhere testing. The trend is for test centre use to decline, but it won’t happen overnight!

John: If another organization is thinking of moving towards certification in the cloud, what advice would you give them?

Ralf: Ensure that you are aware of the challenges I mentioned and can deal with them. And do some pilots before you try to scale.

Interested in learning more about Online Proctoring? I will be presenting a session on ensuring exam integrity with online proctoring at Questionmark Conference 2016: Shaping the Future of Assessment in Miami, April 12-15. I look forward to seeing you there! Click here to register and learn more about this important learning event.

Will 2016 mark the end of test centers?

John Kleeman HeadshotPosted by John Kleeman

The Internet is causing disruptive change in many industries. When I was a child, we had a bookshelf full of encyclopaedias, nowadays my children can find out much more than was ever in these large books on Google or Wikipedia. This is just one of many disruptions the Internet has brought.

Building being demolishedCurrently many high-stakes exams are taken at “bricks and mortar” test centers. Participants need to travel to a test center to take the exam, often having to journey a considerable distance; tests need to be scheduled for a fixed slot in advance; and participants or their employer must pay a substantial fee for use of the test center. Increasingly, the Internet is allowing a new dynamic, with participants taking tests at their offices or homes, with a remote, online proctor ensuring integrity.

So will the Internet take another scalp? Will 2016 be the year that test centers for high stakes tests and exams start to disappear? Will there even be bulldozers demolishing test centers or re-purposing them for other tasks?

Interested in learning more about online proctoring? I will be presenting a session on Ensuring Exam Integrity with Online Proctoring at the Questionmark Conference 2016: Shaping the Future of Assessment in Miami, April 12-15. I look forward to seeing you there! Click here to register and learn more about this important learning event. 

It may be a while before the bulldozers move in, but as shown in my previous post Online or test center proctoring: Which is best?, there are many advantages of online proctoring. Questionmark exams can be delivered both at test centers and via online proctoring, but I am increasingly coming to believe that online proctoring is the future for high stakes exams.

The benefit to the participant of not having to travel to a test center and being able to take the exam in a flexible timeslot is immense. To illustrate this, here is a quote from an independent blog entry by a candidate describing his experience taking an SAP certification exam in 2015:

“I was able to take my test from my home office. This is such a big improvement because I did not have to drive to a test centre which in my case is far away and where I can only take the test on specific days in the year.”

The main reason that organizations use test centers is that it feels more secure. But as I suggested in Online or test center proctoring: Which is more secure?, there are arguments on both sides for security, and most organizations probably use test centers rather than online proctoring because it’s what they have been doing for a long time, rather than necessarily because it is safer.

Undercover picture fromm BBC of test cheatingThere is a long history of security breaches at test centers, and some of the most flagrant happen due to proctors or invigilators helping participants pass the test. The BBC recently reported on several cases of cheating within construction industry exams at test centers in the UK – see here for a text summary and here for an excellent 13 minute video including undercover footage of the cheating. In one case, a proctor took the exam for the participant; in another case, a roomful of people were taking the exam and someone was showing the answers on a large screen at the front of the room (see picture to the right); and in a third case, participants answered the exam but didn’t submit, and the proctor reviewed and improved the answers before submitting.

This is just one of many, many examples. When participants see a financial benefit to passing a high stakes test, there are challenges in maintaining the integrity of a face-to-face proctoring system. It’s not impossible for online proctors to be corrupt, but when separated by distance, it is much harder.

The testing industry is very conservative. There will still be exams taken in test centers for many years to come. But I do think that 2016 may well be the beginning of the end for “bricks and mortar” test centers — and that during 2016, the mainstream view will start to think that online proctoring is the future.

Interested in learning more about online proctoring? I will be presenting a session on Ensuring Exam Integrity with Online Proctoring at the Questionmark Conference 2016: Shaping the Future of Assessment in Miami, April 12-15. I look forward to seeing you there! Click here to register and learn more about this important learning event. 

For more on online proctoring, check out this informational page and video below:

Next Page »