Infographic: Online or Test-Centre Proctoring?

Julie ProfilePosted by Julie Delazyn

For many exams, candidates are required to travel to brick-and-mortar test centers where proctors (or invigilators) supervise the process; However, a new way of proctoring certification exams is rapidly gaining traction. Two of the world’s largest software companies, SAP and Microsoft, offer online proctoring for their certification programs, and many other companies are looking to follow suit.

Do you need to understand the key differences and benefits? Here’s an infographic that explains some of the pros and cons of the two approaches.

Proctoring Infographic

For more on online proctoring, check out this informational page and video below:

 

 

Certification in the Cloud and the Move to Online Proctoring: An interview with SAP’s manager of global certification

John Kleeman Headshot

Posted by John Kleeman

I recently interviewed Ralf Kirchgaessner, SAP’s manager of global certification, about how the cloud is changing SAP certification. This is a shortened version of my conversation with Ralf. To read the full previously published post, check out this SAP blog.

John: What are the key reasons why SAP has a certification program?

Ralf: The overall mission of the program is that every SAP solution should be implemented and supported ideally by a certified SAP resource. This is to ensure that implementation projects go well for customers, and to increase customer productivity while reducing their operating costs. Customers value certification. In a survey of SAP User Group customers in Germany and the US, 80 percent responded that it was very important to have their employees certified and over 60 percent responded that certification was one of the criteria used to select external consultants for implementation projects.

John: What important trends do you see in high tech and IT certification?

Ralf: What comes first to the mind is the move to the cloud. Throughout the technology industry, the cloud drives flexibility and making everything available on demand. One aspect of this is that release cycles are getting quicker and quicker.

For certification, this means that consultants and others have to show that they are always up to date and are certified on the latest release. It’s not enough to become certified once in your lifetime: you have to continually learn and stay up to date. But of course if you are taking certification exams more often, certification costs have to be much lower. In some regions, people have to travel large distances to get to a test centre. With more frequent certification, it’s not practical to travel to a testing centre every time you take a certification. So our aim is to allow certification anytime and anywhere using the cloud.

John: How does online proctoring work for the candidate?

Ralf: A remote proctor monitors the candidate via a webcam, and there are a lot of security checks done by the proctor and by the system. For example, a secure browser is used, the candidate has to do a 360 degree check of his or her room, and there are lots of specific controls. For instance, you aren’t allowed to read the questions silently with your lips in case someone is watching or listening.

The great advantage to the candidate is flexibility. If someone says, “I’d like to do my exam in the middle of the night or on weekends because during the week I’m so busy with my project,” they can. They might say that they’d like to do their exam on Saturday afternoon: “After spending two hours playing with my kids, I’m relaxed to do my exam!” It’s such a flexible way to get certified and to quickly demonstrate that they have up-to-date knowledge and are allowed to provision customer systems.

John: Who benefits from certification in the cloud? Candidates, customers, partners or SAP?

Ralf: Of course, I think all benefit! Candidates have flexibility and lower cost. Customers can be sure that partner consultants who work for them are enabled and up to date. For partners, it’s a competitive advantage to show that their consultants are up to date, especially for new technologies like S/4HANA and Simple Finance. A partner is much more likely to be chosen to deploy new technologies if they can demonstrate that they have several consultants already certified in something that’s just been released. And for SAP, our goal is to have engaged consultants, happy partners and lower support costs. So everyone genuinely benefits.

John: What are some of the challenges?

Ralf: One example is that it’s important in cloud certification to get data protection right. SAP have very detailed requirements that we ensure our vendors like Questionmark meet.

Security is also a challenge. You need to prevent cheating and stealing questions.  And interfaces and integration need to be right. We have worked out how we get the data from our HR systems, how people book and subscribe to exams and then how they can authenticate with single sign-on into the certification hub to take cloud exams.

The delta concept also gives challenges. You need very precise pre-requisite management logic, where the certification software checks for example that, if you want to take the delta exam, you have already passed the core exam. It also can sometimes be difficult to prepare a good delta exam, particularly if a new release has very specific or detailed features, including some that apply in only some industries.

Lastly, providing seamless support is a challenge when using multiple vendors. The candidate doesn’t care where a problem happened: he or she just wants it fixed.

John: Where do you see the long term future of high-tech certification? Will there still be test centres, or will all certification be done via the cloud?

Ralf: Test centres won’t disappear at once, but there is a trend of moving from classroom-based learning and testing to learning and certification in the cloud. The future will belong to anytime, anywhere testing. The trend is for test centre use to decline, but it won’t happen overnight!

John: If another organization is thinking of moving towards certification in the cloud, what advice would you give them?

Ralf: Ensure that you are aware of the challenges I mentioned and can deal with them. And do some pilots before you try to scale.

Interested in learning more about Online Proctoring? I will be presenting a session on ensuring exam integrity with online proctoring at Questionmark Conference 2016: Shaping the Future of Assessment in Miami, April 12-15. I look forward to seeing you there! Click here to register and learn more about this important learning event.

Online or test center proctoring: Which is best?

John Kleeman HeadshotPosted by John Kleeman

A new way of proctoring certification exams is rapidly gaining traction. This article compares and contrasts the old with the new.

Many high-tech companies offer certification exams for consultants, users and implementers. Such exams often require candidates to travel to a bricks-and-mortar test center where proctors (or invigilators) supervise the process.

Now, however, online proctoring is becoming prevalent: each candidate takes the exams at his or her home or office, with a proctor observing via video camera over the Internet. Two of the world’s largest software companies, SAP and Microsoft, offer online proctoring for their certification programs, and many other companies are looking to follow suit. This article explains some of the pros and cons of the two approaches.

workplace_addFactors for choosing online proctoring

  • Reduced travel time.  Candidates can take an exam without wasting time traveling to a test center. This is an important saving for their employers – often the test sponsor’s customers.
  • Convenient scheduling. A candidate can choose a convenient time, for example after the kids have gone to bed or when work pressures are lowest. Usually one needs to book in advance to attend a test center, but it’s often possible to schedule an online proctor at short notice.
  • Fairness. With an exam at a test center, some people will have had a short journey and others a longer one. Some might have experienced a traffic jam or other hassle getting there. This gives an advantage to those who happen to live closer, as they will have less anxiety. An online experience reduces the variability of the exam experience.
  • Accessibility. Candidates take online proctored exams on their own computers, using their normal accessibility aids such as screen readers or special input devices, whereas these require setup at a test center. Some test centers only provide their own (often limited) tools for providing accommodations, so candidates are working with unfamiliar tool sets. This places them at a disadvantage. Also, for people with certain disabilities, travel is a major inconvenience.
  •  Keeping certifications up to date. If candidates have to travel to a test enter, a test sponsor can’t realistically require an exam to be taken more than once every few years. But in today’s world, products and job skills change very quickly, so certification risks being out of date. The availability of online  proctoring allows update exams (assessing candidates on what has changed since their last exam) to be taken as products change, which makes the programme more valid.
  • Greater authenticity. The more authentic assessments are, the more they measure actual performance. See Will Thalheimer’s excellent paper on measuring learning results for more on this. Assessing someone in their work environment with online proctoring is more authentic and so will likely measure performance better than putting them in a test center.

office-buildingFactors for choosing test center proctoring

  • Standardized computers. While online proctoring requires the candidate to have an appropriate computer, internet connection and webcam that they know how to use, test centers provide a computer that is already set up. For most certification programmes, it’s fair and reasonable that candidates use their own computers (often called BYOD – Bring Your Own Device). But for some programmes, this might be less fair. For example, in professions where IT literacy is not required, it might not be fair to expect people to have access to a PC with webcam that they know how to use.
  • Very long exams.  In online-proctored exams, the candidate is usually forbidden from taking a break for security reasons.  Most exams can be taken in one sitting, but if exams are longer than three hours, a test center makes sense.
  • Regulation. Some regulators or government authorities may require delivery of an exam with a physically present proctor at a test center.
  • Geographical convenience.  In some cases, test centers may be close at hand. For example, a university might have all its candidates already present, or, for some test sponsors, candidates may all live in metropolitan areas close to test centers.

checkOther factors to consider:

  • Language. In theory, a candidate could schedule an online proctor in his or her own language, though in practice many programs only offer English-speaking proctors. A test center may well not have proctors who can speak different languages, but typically will speak the local language.
  • Security. You might think that the security is stronger in a test center than with online proctoring. However, over the years there have been many incidents where face-to-face proctors have coached candidates. Online proctoring also makes it feasible to administer exams more frequently, which helps security by making impersonation harder. This is a big subject, and I’ll follow up with a blog post about security.

I’d welcome your thoughts on any other factors for and against online proctoring.