Certification in the Cloud and the Move to Online Proctoring: An interview with SAP’s manager of global certification

John Kleeman Headshot

Posted by John Kleeman

I recently interviewed Ralf Kirchgaessner, SAP’s manager of global certification, about how the cloud is changing SAP certification. This is a shortened version of my conversation with Ralf. To read the full previously published post, check out this SAP blog.

John: What are the key reasons why SAP has a certification program?

Ralf: The overall mission of the program is that every SAP solution should be implemented and supported ideally by a certified SAP resource. This is to ensure that implementation projects go well for customers, and to increase customer productivity while reducing their operating costs. Customers value certification. In a survey of SAP User Group customers in Germany and the US, 80 percent responded that it was very important to have their employees certified and over 60 percent responded that certification was one of the criteria used to select external consultants for implementation projects.

John: What important trends do you see in high tech and IT certification?

Ralf: What comes first to the mind is the move to the cloud. Throughout the technology industry, the cloud drives flexibility and making everything available on demand. One aspect of this is that release cycles are getting quicker and quicker.

For certification, this means that consultants and others have to show that they are always up to date and are certified on the latest release. It’s not enough to become certified once in your lifetime: you have to continually learn and stay up to date. But of course if you are taking certification exams more often, certification costs have to be much lower. In some regions, people have to travel large distances to get to a test centre. With more frequent certification, it’s not practical to travel to a testing centre every time you take a certification. So our aim is to allow certification anytime and anywhere using the cloud.

John: How does online proctoring work for the candidate?

Ralf: A remote proctor monitors the candidate via a webcam, and there are a lot of security checks done by the proctor and by the system. For example, a secure browser is used, the candidate has to do a 360 degree check of his or her room, and there are lots of specific controls. For instance, you aren’t allowed to read the questions silently with your lips in case someone is watching or listening.

The great advantage to the candidate is flexibility. If someone says, “I’d like to do my exam in the middle of the night or on weekends because during the week I’m so busy with my project,” they can. They might say that they’d like to do their exam on Saturday afternoon: “After spending two hours playing with my kids, I’m relaxed to do my exam!” It’s such a flexible way to get certified and to quickly demonstrate that they have up-to-date knowledge and are allowed to provision customer systems.

John: Who benefits from certification in the cloud? Candidates, customers, partners or SAP?

Ralf: Of course, I think all benefit! Candidates have flexibility and lower cost. Customers can be sure that partner consultants who work for them are enabled and up to date. For partners, it’s a competitive advantage to show that their consultants are up to date, especially for new technologies like S/4HANA and Simple Finance. A partner is much more likely to be chosen to deploy new technologies if they can demonstrate that they have several consultants already certified in something that’s just been released. And for SAP, our goal is to have engaged consultants, happy partners and lower support costs. So everyone genuinely benefits.

John: What are some of the challenges?

Ralf: One example is that it’s important in cloud certification to get data protection right. SAP have very detailed requirements that we ensure our vendors like Questionmark meet.

Security is also a challenge. You need to prevent cheating and stealing questions.  And interfaces and integration need to be right. We have worked out how we get the data from our HR systems, how people book and subscribe to exams and then how they can authenticate with single sign-on into the certification hub to take cloud exams.

The delta concept also gives challenges. You need very precise pre-requisite management logic, where the certification software checks for example that, if you want to take the delta exam, you have already passed the core exam. It also can sometimes be difficult to prepare a good delta exam, particularly if a new release has very specific or detailed features, including some that apply in only some industries.

Lastly, providing seamless support is a challenge when using multiple vendors. The candidate doesn’t care where a problem happened: he or she just wants it fixed.

John: Where do you see the long term future of high-tech certification? Will there still be test centres, or will all certification be done via the cloud?

Ralf: Test centres won’t disappear at once, but there is a trend of moving from classroom-based learning and testing to learning and certification in the cloud. The future will belong to anytime, anywhere testing. The trend is for test centre use to decline, but it won’t happen overnight!

John: If another organization is thinking of moving towards certification in the cloud, what advice would you give them?

Ralf: Ensure that you are aware of the challenges I mentioned and can deal with them. And do some pilots before you try to scale.

Interested in learning more about Online Proctoring? I will be presenting a session on ensuring exam integrity with online proctoring at Questionmark Conference 2016: Shaping the Future of Assessment in Miami, April 12-15. I look forward to seeing you there! Click here to register and learn more about this important learning event.

Tips for preventing cheating and ensuring assessment security

julie-smallPosted by Julie Chazyn

Last week I wrote about tips for protecting  intellectual property.  It’s equally important keep people from cheating on tests, so here are three pointers on that subject.  I’ll be following this up with more tips in future posts.  Leave me your comments; we can always add to lists like these!

.Screening tests

Consider givinig a small pre-screening test to prevent people from taking an assessment that is beyond their current ability level. If a participant can‘t answer a certain number of these questions correctly they will not be allowed to see the remainder of the assessment. When the time does come for them to take the test,  they will not have already seen its content.

Candidate agreements

Candidate agreements or examination honor codes require  a participant’s agreement before they start an assessment, say by clicking on an “OK “ or  “Yes” button after reading the exam’s code of conduct.

The code might say something like this:  “I agree to answer the questions on this assessment without obtaining assistance from another person or via electronic means. I agree to not to share my answers with anyone during or after the exam. I further agree to not memorize or otherwise steal the intellectual property contained in this exam. I accept that if any of these conditions are violated, my exam results will be set to a zero, I will not be able to retake the exam for a period of 10 years, and I may be charged with a crime under regional laws.”

Here are some topics you might want to cover in a candidate agreement:

  • The test vendor will have the option to terminate the assessment if suspicious behavior is detected
  • The candidate must abide by the rules of the test center, organization, or program
  • The candidate will not provide false ID or false papers
  • The candidate cannot take the test on behalf of someone else
  • The candidate will not engage in cheating in any form
  • The candidate will not help others cheat by disclosing information about the assessment
  • The candidate will not use aids that are not allowed
  • The candidate will not solicit someone else to take the test
  • The candidate will not cause a disturbance in the testing center
  • The candidate will not tamper with the test center in any way
  • The candidate will not share information

Limiting content exposure/leakage

In order to limit the amount of question content being shown to a participant at any given time, think about using question-by-question templates. These present questions one at a time to participants so that exam content is not completely exposed on screen. Participants who may intend to take pictures of the exam content or otherwise steal intellectual property will not be able to do so all at once.

There are many fine resources for learning how to prevent cheating. Two of thes are books by Dr. Gregory Cizek:  Cheating on Tests: How to Do It, Detect It and Prevent It and Detecting and Preventing Classroom Cheating: Promoting Integrity in Assessment.

There’s also our white paper: “Delivering  Assessments Safely and Securely,” and of course this blog! Watch for more security tips in my future posts.