How is the SAP Global Certification program going? A re-interview with SAP’s manager of global certification, part 2.

Posted by Zainab Fayaz

This is the second part of the two-part interview between, John Kleeman, Founder and Executive Director at Questionmark and Ralf Kirchgaessner, Manager of the SAP Global Certification program. This is a continuation of the use of Questionmark software in their Certification in the Cloud program. You can read the first part here. In the second part of the interview, John asks questions about the business benefits of certification and what advice Ralf has for other organizations.

John: What are the business benefits to SAP of certification?

Ralf: There are many benefits to the SAP Global Certification. So, let’s begin from the individual learner’s perspective.

Firstly, earning the SAP Global Certification increases your personal value; not only does it drive personal development; which often leads to increased responsibilities and promotion within your organization, but it also showcases and proves that you stay current and update your skills to the latest releases. Additionally, since 2018, professionals can gain wider recognition through sharing their SAP Global Certification digital badges.

SAP Global Certification is of great value not only for individuals but also for consultancies in the SAP ecosystem. SAP Global Certifications provide a clear measure of a company’s organizational capabilities, which give a competitive advantage, especially if the company has certified professionals in new and innovative areas, like SAP C/4HANA Cloud.

John: What about the customers? What benefits are there for them?

Ralf: Indeed, the most important benefit is the value for our customers. If SAP can ensure that the consultancy eco-system is well enabled and certified, it helps reduce the total cost of ownership (TCO) and ensures successful implementation costs. And in the end, this is of course also important for SAP, as this helps to increase the adoption of our software and reduces implementation risks.

John: Tell me a bit more about the recently introduced digital badges for people who get certified that you just mentioned. How useful is that?

Ralf: The introduction of digital badges for SAP Global Certification has been an absolute success! Making your workforce visible on the market is important and by sharing the digital badge proves that the workforce is currently in their knowledge. If on LinkedIn, you search for ‘certified SAP consultants’, you would find thousands of shared badges. Digital badge claim rates beyond industry standards show that people waited with much anticipation to share their achievements digitally.

We are constantly looking for ways to improve our services and with the help of Questionmark, going forward we will be able to issue badges, even faster. In the near future, once candidates have passed their SAP Global Certification exam this will trigger the issuing of badges in “real-time”!

We have reached our ultimate goal and an overall mission of our certification programme if customers ask consultants for their digital badges to show their SAP Global Certification status.

John: There seems a slow move across the community from test centers to online proctoring. I know that for SAP, you deliver some exams in your offices but most in the cloud with online proctoring. How do you see this changing in the industry in general? Will all IT exams be done by online proctoring one day soon?

Ralf: SAP very much uses the model of taking exams wherever and whenever it is most convenient. Nevertheless, we use one harmonized infrastructure, for all our exams and these can be taken at our offices, in classrooms or in the cloud.

I think much of this evolves from the changing landscape in learning behaviors and offerings. In terms of the advantages of using test centers and online proctoring; there is a legitimate reason for test centres to exist; as there are groups of people who will still want to learn together – in one place at one time. However, as the shift moves towards a rise in remote learning, both synchronous (live virtual classrooms) and asynchronous, which are supported by social and peer learning via online learning rooms, then of course, online proctoring will become more popular.

John: What advice would you give to other high-tech companies who are thinking of setting up or improving their certification program?

Ralf: Two things instantly come to mind – online proctoring and digital badging. Certification programs that do not use online proctoring and digital badging should urgently consider improving their program as the benefits of implementing both features are tremendous.

More on certification
Interested in learning more about certification programs?  Find out how you can build your own certification program in 10-easy steps.

 

How is the SAP Global Certification program going? A re-interview with SAP’s manager of global certification, part 1.

Posted by Zainab Fayaz

Back in 2016, John Kleeman, Founder and Executive Director of Questionmark interviewed Ralf Kirchgaessner, Manager of SAP Global Certification program about their use of Questionmark software in their Certification in the Cloud program and about their move to online proctoring. You can see the interview on the Questionmark blog here. We also thought readers might be interested in an update, so here is a short interview between the two on how SAP are getting on three years later:

John: Could you give us an update on where you are with the Certification in the Cloud program?

Ralf: The uptake, adoption and increase of Certification in the Cloud is tremendous! Over the years we have seen a significant increase in the volume of candidates taking exams in the cloud; the numbers doubled from 2016 to 2017 and increased almost by 60% in 2018. This means more than 50% of SAP Global Certification exams are now done remotely!

John: Are all your SAP Global Certification exams now available online in the cloud?

Ralf: Nearly so. By mid-2019 we plan on having the complete portfolio of every SAP exam available on the cloud. This is great news for our learners who have invested in a Certification in the Cloud subscription. So, we then have Certification in the Cloud not only for SAP SuccessFactors and SAP Ariba, but for all products, including SAP C/4HANA.

John: How many different languages are your exams translated into?

Ralf: This depends on the portfolio. Some of our certifications are available in English and others, such as for SAP Business One are translated in up to 20 languages.

John: How are you dealing with the fast pace of change within SAP software in a certification context? How do you ensure certifications stay up to date when the software changes?

Ralf: This is of course a challenge. In previous years, it was the case of getting certified once every few years. However, now you must keep your skills up-to-date and stay current with quarterly release cycles of our SAP Cloud solutions. Also, for people who are first timers or newly enter the SAP eco-system; it is important that they are certified on the latest quarterly release.

To help overcome this challenge, we have developed an agile approach to updating our exams; we use the Questionmark platform for those who are new to the eco-system to help them getting certified initially. We also have a very good process in place and often use the same subject matter experts when it comes to keeping up to the speed of software changes.

For already certified professionals, another way to remain up to date is through our ‘Stay Current’ program. For some of our solutions, partners have to come back every 3 months to show that they are staying current. They do this in the form of taking a short “delta” knowledge assessment. For instance, for certified professionals of SAP SuccessFactors it is mandatory to stay current in order to get provisioning access to the software systems.

In 2018, SAP’s certification approach was acknowledged with the ITCC Innovation Award. Industry peers like from Microsoft, IBM and others recognized this great achievement with this award.

 

Q&A: Sue Martin and John Kleeman discuss steps to building a certification program

Posted by Zainab Fayaz

Certification programs are a vital way of recognizing knowledge, skills and professional expertise, but, during a time of digital transformation, how do you build a program that is sustainable and adaptable to the evolving needs of your organization, stakeholders and the market?

Questionmark Founder and Executive Director, John Kleeman and Sue Martin, certification expert and Business Transformation Consultant will present the upcoming webinar Building a Certification Program in 10 easy stepsat 4pm (GMT) / 11am Eastern / 5pm CET on 12 February 2019. However, before then, we wanted to find out a little bit more about what they’ll be covering during the session, so we’ve interviewed the two experts to gain insight!

Tell us a bit about what you’ll be covering during the webinar:

Sue: During the webinar, we’ll be covering a range of things; from the conceptual steps of building a certification program to the many projects that have evolved from these and the importance of outlining key steps from the very beginning of the process for creating a comprehensive and cohesive certification program.

We will also talk about the value certification program, can add to an organization, not only in the short-haul but also for many years to come. It is important to remember, “why” and “what” you are trying to achieve, and this webinar will provide detail on how the alignment of strategic goals and communication with stakeholders contributes to the success of an adaptable certification program.

John: We’ll be discussing a range of things during the webinar, but here are the ten easy steps that we’ll be describing:

  1. Business goals
  2. Scope
  3. Security
  4. Vendor evaluation
  5. Blueprint and test design
  6. Test development
  7. Pilot
  8. Communications
  9. Delivery
  10. Reporting and monitoring

What influenced the selection of these 10-steps you have identified in building a certification program?

John:  Sue and I sat down to plan the webinar when we were together at the OEB conference in Berlin in December. Although we wanted to cover a bit some of the obvious things like test design and development, we wanted to make sure people think first about the preparation and planning, for example getting organization buy-in and working out how to market and communicate the program to stakeholders. So we’ll be focusing on what you need to do to make a successful program, and that will drive everything you do

Although you’ll be covering the key steps for building a certification program during the webinar, can you advise on three key steps you find to be the most important during the process:

Sue:
1. Planning:
The emphasis of the program’s work should be at the start, in the planning phase – especially in order to build a flexible program which will adapt to the needs of your audience and stakeholders as their needs change over time. In all of the individual project components, whether it be test creation, vendor evaluation or communications rollout, for example, design and plan for the end goal. For example, when it comes to creating an exam, you plan for it right at the start of the project – you hit the ground running! It is not all about item writing, but also the development of the project from the beginning and if you don’t plan; this can lead to the lack of validity in the exam program and inconsistency over time

2. Practical tips and tricks for approaching various elements of your program development: It is important to set out the target audience; identify their learning journey and how they learn – in knowing this, can you go forward and build a certification program that can become integrated and aligns with the learning process

3. Scope: This is very important; setting the scope is a priority. Of course, in the greater scheme of things; you’ll have a mission statement, which provides you with a strategic vision, but when it comes to the finer detail and knowing what countries to enter, the pricing structure or knowing whether to offer remote proctoring; always keep in mind three things: the value contribution, the stakeholders and ask yourselves “yes, but why?”; as this will help align with organizational objectives.

What can attendees take away from the webinar you’ll present?  

Sue: Those attending will learn the value and importance of planning and questioning everything from the start of the process. We’ll share advice on the importance of having a value statement for every part of the process and making sure you know that a certification program is what you are looking for. By attending you can walk away with knowing the operational and strategic steps you must go through in order to build a program that is sustainable; think of it as a checklist!

John: If you’re starting a new certification program, I think this webinar will help guide you and help you create it more easily and more effectively. And if you already have a certification program and want to improve it, you’ll probably be doing a lot of what we suggest already but I hope they’ll be something for everyone to take away and learn.

Want to know more?

If you’re interested in learning more about the steps to building a certification program that meets the needs of your organization and stakeholders; then join John and Sue deliver a webinar session dedicated to Building a Certification Program in 10 easy steps on 12 February 2019 at 4pm (GMT) / 11am Eastern / 5pm CET.

You can register here.

A little bit more about our two experts:

John Kleeman is Executive Director and Founder of Questionmark. He has a first-class degree from Trinity College, Cambridge, and is a Chartered Engineer and a Certified Information Privacy Professional/Europe (CIPP/E). John wrote the first version of the Questionmark assessment software system and then founded Questionmark in 1988 to market, develop and support it. John has been heavily involved in assessment software for 30 years and has also participated in several standards initiatives including IMS QTI, ISO 23988 and ISO 10667. John was recently elected to the Association of Test Publishers (ATP) Board of Directors.

Sue Martin is a trusted advisor to companies and institutions across Europe in the area of workforce credentialing, learning strategies and certification. Her career prior to consulting included a role as Senior Global Certification Director for SAP and several regional and global management roles in the testing industry. She has also held several positions within industry institutions, such as the Chair of the European Association of Test Publishers and is currently a member of the Learning & Development Committee at BCS (British Computer Society).

 

International University gains scalability, advanced authoring, single sign-on and more!

ChloePosted by Chloe Mendonca

Earlier this month I had the pleasure of speaking with The International University of Languages and Media (IULM) based out of Milan, Italy. IULM has been using Questionmark’s Assessment Management System for almost 10 years, so I was also keen to see what prompted their recent move from managing their assessment program on-premise to Questionmark OnDemand (Questionmark’s SaaS cloud-based solution).iulm

IULM serves approximately 5,000 students enrolled across a range of postgraduate and undergraduate courses including interpretation, translation and cultural studies; communication, public relations and advertising, and arts and tourism.

Patrizia Lettieri, the digital platform administrator in the university’s IT department explains why after 10 years, the IULM decided to make the move:  “Our teachers liked using Questionmark; we didn’t want to change that, but there was definitely a need for a more scalable and innovative solution that would match the direction the university was headed in terms of growth. Security and reporting were also very important to us and there is nothing on par with Questionmark OnDemand in regards to security and reporting.”

Here are some highlights from our conversation. You can read the entire case study here.

Advanced authoring
A big draw for IULM in using Questionmark was the ease of use when it came to assessment authoring and improving the way items are written and also the efficiency with which they can manage their item bank. Patrizia elaborates: “The variety of item types and ability to embed images, videos and links into our questions and feedback enable us to further develop the way we are helping our students to learn. Meta tagging of items is also very simple. Before using OnDemand we had to separate these into different folders and our teachers used to delete items that were invalid. Now we can easily assign the appropriate meta tags and modify the status of questions to retired if they are invalid.”

Scalability and save-as-you-go
iulm_campusBefore using Questionmark OnDemand IULM had a few challenges when it came to assessing large numbers of students.  With OnDemand, IULM can reliably deliver their assessments to multiple classes of students. And in the event that something does go wrong — such as a power cut or other technical failure — Questionmark’s save-as-you-go is there to ensure student answers are automatically saved at regular intervals. “My priority is the students; they are our customers and I want them to be comfortable,” adds Patrizia. “With OnDemand everything runs smoothly and our students are more comfortable and confident when taking their assessments.”

Single Sign-On
It used to be that every day, staff and students at the university were having to interact with a variety of applications requiring individuals to remember different credentials for the various applications.  “With Questionmark OnDemand, we were able to take advantage of single sign-on (SSO). For a forward-thinking technological university, improving access for both students and staff just made sense for us.” SSO also saves IULM time on the administrative side. Once students have completed their studies and leave the university, staff no longer have to remove their access to each application but only need to remove each student’s identity provider account. Without that account, the student can no longer log on to any of the linked applications.

Multilingual delivery
Being an international university, having students who speak different languages is a common scenario. Questionmark’s multilingual delivery helps IULM accommodate students whose first language is not Italian or English by enabling them to change the delivery interface to their preferred language. Having the navigation controls in a participant’s own language can often make students feel more at ease during the test taking experience.

If you’re interested in trying out Questionmark OnDemand, request a demo today and one of our team will show you how simple assessment management can be.

Security, Reporting and Online Testing in Academia: A Q&A with Don Kassner

Julie ProfileI recently spoke with Don Kassner, who has joined Questionmark as vice president of academic markets. I wanted to take a moment to welcome Don and to ask him a few questions about his extensive background, his new job role and his hopes for the future. Here’s a snippet of our conversation:

Don Kassner, VP of academic markets, Questionmark

You have had an extensive career. What is your background, and how has it influenced the insight you have on the learning and assessment world?

At heart, I am an entrepreneur.  I started my first business when I was 8 years old and have been starting and building things ever since. In the last 15 years or so I have focused on technology, training and education. During that time, I have served as the CFO for an auto dealership training company and as president of a small university in addition to founding and building the largest online proctoring company. I have also served on the faculty of San Jose State University (economics) and have been an active member of multiple onsite accreditation evaluation teams.

Now that you are part of the Questionmark team, can you share a little about your new role and the goals you have for this position?

I am excited to reach out to the academic markets and address ongoing concerns around testing.

Colleges and universities want to deliver tests and exams that are consistent, fair, reliable and defensible. They want to deliver and monitor course evaluations, identify knowledge gaps and place students in appropriate courses, and be able to analyze questions to determine which ones are valid and fair. They want to do this while increasing student satisfaction and reducing cheating.

With Questionmark’s extensive security, reporting and analytics tools, that’s all possible.  My focus is to leverage my insight and experience and put this secure and easy-to-use tool in the right hands.

Can you share some of the important topics surrounding the academic markets that are on your radar?

In academia, there are real issues related to online testing. These are the top two:

  • Is the student doing the work? – With technology comes flexibility and as the stakes become higher, students will naturally look for ways to enhance their scores.
  • Are the exam results fair?  Did all the students face the same conditions with the same opportunities and were the exam rules followed?

Questionmark OnDemand, especially its analytics and reporting tools, can provide reports that can uncover the meaning hidden within assessment results, such as an item or assessment’s reliability and defensibility. In regards to security, Questionmark Secure’s lock-down browser plays a huge role in helping organizations provide a secure environment in which to deliver high stakes assessments. This can significantly help reduce the risk of cheating when deployed along with other defenses to combat impersonation and content theft.

We’re very excited to have you on board! What are your plans for keeping in touch with us on The Questionmark Blog?

Once a teacher, always a teacher! I look forward to turning back the clock to my university days and speaking on the hot-button topics that surround academic testing. My experience in education and entrepreneurship will help me as I share my thoughts on current market trends and the future of assessment and testing.

Thank you Don for taking the time to speak to us today!

You can follow Don on Twitter and connect with him on LinkedIn

Online Proctoring: FAQs

John Kleeman HeadshotPosted by John Kleeman

Online proctoring was a hot-button topic at Questionmark’s annual Users Conference. And though we’ve discussed the pros and cons in this blog and even offered an infographic highlighting online versus test-center proctoring, many interesting questions arose during the Ensuring Exam Integrity with Online Proctoring  session I presented with Steve Lay at Questionmark Conference 2016.

I’ve compiled a few of those questions and offered answers to them. For context and additional information, make sure to check out a shortened version of our presentation. If you have any questions you’d like to add to the list, comment below!

What control does the online proctor have on the exam?

With Questionmark solutions, the online proctor can:

  • Converse with the participant
  • Pause and resume the exam
  • Give extra time if needed
  • Terminate the exam

What does an online proctor do if he/she suspects cheating?

Usually the proctor will terminate the exam and file a report to the exam sponsor.

What happens if the exam is interrupted, e.g. by someone coming in to the room?

This depends on your security protocols. Some organizations may decide  to terminate the exam and require another attempt. In some cases, if it seems an honest mistake, the organization may decide that the proctor can use discretion to permit the exam to continue.

Which is more secure, online or face-to-face proctoring?online proctoring

On balance, they are about equally secure.

Unfortunately there has been a lot of corruption with face-to-face proctoring, and online proctoring makes it much harder for participant and proctor to collude as there is no direct contact, and all communication can be logged.

But if the proctors are honest, it is easier to detect cheating aids in a face-to-face environment than via a video link.

What kind of exams is online proctoring good for?

Online proctoring works well for exams where:

  • The stakes are high and so you need the security of a proctor
  • Participants are in many different places, making travel to test centers costly
  • Participants are computer literate – have and know how to use their own PCs
  • Exams take 2-3 hours or less

If your technology or subject area changes frequently, then online proctoring is particularly good because you can easily give more frequent exams, without requiring candidates to travel.

What kind of exams is online proctoring less good for?

Online proctoring is less appropriate for exams where:

  • Exams are long and participants needs breaks
  • Exams where participants are local and it’s easy to get them into one place to take the exam
  • Participants do not have access to their own PC and/or are not computer literate

How do you prepare for online proctoring?

Here are some preparation tasks:

  • Brief and communicate with your participants about online proctoring
  • Define clearly the computer requirements for participants
  • Agree what happens in the event of incidents – e.g. suspected cheating, exam interruptions
  • Agree what ID is acceptable for participants and whether ID information is going to be stored
  • Make a candidate agreement or honor code which sets out what you expect from people to encourage them to take the exam fairly

I hope these Q&A and the linked presentation are interesting. You can find out more about Questionmark’s online proctoring solution here.