Learning, Training and Assessments in Regulatory Compliance – Implementation Best Practices

Posted by John Kleeman

I’m pleased to let you know of a new joint SAP and Questionmark white paper on implementation best practices for learning, training and assessments in regulatory compliance. You can download the white paper here.

There has been a huge change in the regulatory environment for companies in the last few years. This is illustrated nicely by the graph below showing the number of formal warning letters the U.S. Food and Drug Administration (FDA) issued in the period from 2010 to 2016 for various compliance infractions.

Rise in FDA warning letters from 2010 to 2016, numbers increase from a few hundred a year to over 10,000 a year

Of course it’s not just letters that regulators issue, there have also been huge increases in the fines issued on companies in areas including rules breaches in banking, data protection, price-fixing and manufacturing.

Failure to effectively train or assess employees is a significant cause of compliance errors, and this white paper authored by SAP experts Thomas Jenewein, Simone Buchwald and Mark Tarallo and me (Questionmark Founder and Executive Director, John Kleeman) explains how technology can help address the issue.

The white paper starts by looking at key factors increasing the need for training, learning, and assessments to ensure that businesses stay compliant and then goes on to consider three drivers for compliance learning – Organization Imposed, Operations Critical and Regulatory. The white paper then looks at how

  • A Learning Management System (LMS) can manage compliance learning
  • Learning Content and Documentation Authoring Tools can author compliance learning
  • An Assessment Management System can be used to diagnose the training needed, to help direct learning and to check competence, knowledge and skills.

A typical LMS includes basic quiz and survey capabilities, but when making decisions about people, such as whether to promote, hire, fire, or confirm competence for compliance or certification purposes, companies need more. The robust functionality of an effective assessment management system allows organizations to create reliable, valid, and more trustworthy assessments. Often, assessment management systems and LMSs work together, and test-takers will often be directed to take assessments by using a single sign-on from the LMS.

The white paper describes how companies can use SAP SuccessFactors Learning, SAP Enable Now and Questionmark software work together productively to help companies manage and deliver effective compliance learning, training and assessments – to mitigate regulatory risk. It goes on to describe some key trends in compliance training and assessments that the authors see going forwards, including how cybersecurity and data protection are impacting compliance.

The white paper is a quick, easy and useful read – you can download it here.

New white paper: Questionmark and Microsoft Office 365

Posted by John Kleeman

I’m pleased to inform you of a new white paper fresh off the press on Questionmark and Microsoft Office 365.

Office logoThis white paper explains how Microsoft Office 365 complements the Questionmark OnDemand assessment management system; and how you can use Office 365 to launch Questionmark surveys, quizzes, tests and exams, how to consume Office 365 resources within Questionmark, and how Office 365 can help analyze results from assessments. You can download the white paper here.

The white paper also describes some of the reasons that organizations use assessments and why it is important for assessments to be valid, reliable and trustable.

Launching assessments from Office 365

Being able to call assessments from within Office 365 allows you to closely connect an assessment to content, for example to check understanding after learning. The white paper describes how you can:

  • Call Questionmark assessments from the Office 365 app launcher
  • Launch an assessment from within a Word, Excel or other Office document
  • Embed an assessment inside a PowerPoint presentation
  • Launch or embed assessments from SharePoint
  • Use SAML to have common identities and seamless authentication between Office 365 and Questionmark OnDemand. The benefit of this is that test-takers can login once to Office 365 and then can take tests in Questionmark OnDemand without needing to login again.

Using Office 365 resources within assessments

Illustration of a picture of a video being used inside an assessmentAssessments of competence are in general more accurate when the questions simulate the performance environment being measured. By putting video, sound, graphics and other media within question stimulus, you help put the participant’s mind into an environment closer to how he/she will be when doing a real-world job task. This makes the question more accurate in measuring the performance of such tasks.

To help take advantage of this, a common use of Office 365 with Questionmark OnDemand is  to make media and other resources that you can use within assessments. The white paper describes how you can use Office 365 Video, PowerPoint, SmartArt and other Office 365 tools to make videos and other useful question content.

Using Office 365 to help analyze results of assessments

People have been using Microsoft Excel to help analyze assessment results since the 1980s and the white paper describes some suggestions on how to do that most effectively with Questionmark OnDemand.

Newer Microsoft tools can also be used to provide powerful insight into assessment results. Questionmark OnDemand makes available assessment data in an OData feed, which can be consumed by business intelligence systems like Power BI. OData is an open protocol to allow the creation and consumption of queryable and interoperable data in a simple and standard way. The white paper also describes how to use OData and Power BI to get further analysis and visualizations from Questionmark OnDemand.

 

The white paper is easy to read and gives practical advice. I recommend reading this white paper if your organization uses Office 365 and Questionmark or if you are considering doing so. You can download the white paper (free with registration) from the Questionmark website.  You can also see other white papers, eBooks and other helpful resources at www.questionmark.com/learningresources.

 

Questionmark Conference 2018 – Registration is Open!

Posted by Brian McNamara

We are excited to announce that registration for Questionmark Conference 2018 is now open!

Questionmark customers will gather March 6th – 9th in Savannah, Georgia, for three days of learning, networking and professional development opportunities.   Whether you are a new user or a pro, Questionmark Conference is the ideal place to get vital information, insight and learning about the latest assessment technologies.

Join us at the Westin Savannah Harbor to enjoy southern hospitality with your peers and the Questionmark team to:

  • Learn about new Questionmark features and functions
  • Attend engaging hands-on product training sessions
  • Network with fellow Questionmark users
  • Get valuable tips, insights and best-practice guidance
  • Plan for the future with a sneak peek at the product road map

Registration is now open at special early-bird rates. Visit the conference website today for details!

Questionmark Conference 2018: Assess for Success | March 6-9 in Savannah

Twelve tips to make questions translation ready

Posted by John Kleeman

We all know the perils of mis-translation. My favourite mis-translation is the perhaps apocryphal tale of a laundry in Rome, Italy putting up a sign in English saying “Ladies, leave your clothes here and spend the afternoon having a good time.” With Questionmark having a translation management system to help you translate questions and assessments, here are some good practice tips on writing questions so they will be easy to translate.

1. Avoid questions that assume syntax is the same in all languages, for example fill-in-blank questions that rely on word order. For example, in English, the verb goes in the middle of a sentence but in Turkish and Korean, the verb is usually at the end of a sentence.

2. Also avoid “broken stem questions”, where the stem is an incomplete sentence and the participant must select the most appropriate answer to finish the sentence. That’s likely to be challenging to translate in some languages where the ordering may not make sense.

3. Keep questions simple. Avoid unnecessarily complex text or question stems with redundancy or unnecessary repetition. Such questions are best simplified before you translate them.

4. Avoid metaphors and idiomatic language in general; things like “in small steps” or “disappear into thin air”, could well introduce translation mistakes.

5. Avoid passive voice where you can. Not all languages make it easy to translate this, and it’s usually best to just use active voice.

6. Thoroughly review questions prior to translation to ensure no ambiguity. if the question wording is ambiguous, the translator’s interpretation of the question may not be the same as that of the question author.

7. If you are using a rating scale across many questions, investigate its cultural appropriateness and, if possible, whether it is widely used in the target language.

8. Test items based on nuances of vocabulary, descriptions of emotions or abstract concepts can be hard to translate, as different languages may have different vocabulary connotations.

9. You also need to be aware of the risk that translating a question could cause a question to become obvious due to different words in the target language, like the following Swedish example.

Question that shows a Swedish translation giving the answer to a question by words being the same

10. Avoid using cultural context within question stimulus. If you are presenting a scenario, make it one that is relevant to different cultures and languages. If it is difficult to avoid a culturally marked context, consider preparing good guidelines for translators in which you define what adaptations are encouraged, desirable and ruled out

11. If your question contains a graphic or video, consider if you can remove any text from it and still keep the question meaningful. Otherwise you need to translate the text in the graphic or video in each language.

12. if you are translating items into several languages, it can be cost effective to conduct a translatability assessment on the items before you do the detailed translation. This will alert to possible issues within various language families prior to doing the more substantial work of full translation. A translatability assessment lets you identify and fix issues early and relatively cheaply. See here for a blog from Steve Dept of CApStAn that explains more.

Thanks to Steve Dept for inspiring this blog post with an excellent conference presentation at EATP last year and for helping me write this article. For some more advice on translating and adapting tests, see the International Test Commission Guidelines for Translating and Adapting Tests or the Cross-cultural Survey Guidelines (CCSG), both of which have been recently updated.

I hope this advice helps you be efficient in your translation efforts. For information on Questionmark OnDemand which includes translation management system capabilities, see www.questionmark.com.

Trust Center: Our Commitment to Security

Posted by Julie Delazyn

Our commitment to security is real. Security and reliability are key to both to our success and the success of our customers. That’s why we’ve established our Trust Center — a single place us to feature our different accreditations and validations. In order to highlight Questionmark’s commitment to security, I spoke to Questionmark’s Executive Director and Founder John Kleeman about what security means to Questionmark and how far we go to ensure the protection and privacy of our customer’s data.

What does security mean to Questionmark?

Organizations and companies around the world — companies, universities, and colleges, certification organizations — entrust us with a large amount of confidential data from assessments. We conduct millions of assessments a year and customers depend on us to keep those secure. To Questionmark, security means: Confidentiality, Integrity and Availability of our services for our customers. It’s one of our central aims and goals.

How does Questionmark make security a priority?

You can never 100% ensure security. As you can see in the media, Governments or large well-regarded corporations have had security breaches –so it’s about reducing that risk and making it unlikely that there will be a security issue. The main way in which we do that is by having an information security management system and putting processes in place to look at the risks and put in place controls and take other actions to reduce them.

Anyone can claim to be secure. What sets us apart from some companies in the space is that we don’t just say we’re secure, we get external validation of our security. We’ve recently certified against the ISO 27001 standard, and that’s an involved, grueling audit, which looks at our procedures. By having external validation, we can prove that experts have looked at our security and checked that it’s real — not just that we’re claiming it.

Questionmark is a global company with customers around the world – how do the standards we meet reflect that?

Everyone in the world wants confidentiality, integrity and availability, but there are local compliance criteria—technical rules – that can differ. While ISO 27001 is an international accreditation, we’re also looking to meet various national standards. We’re in the process of establishing compliance with FedRAMP, which is a US Government requirement. We are an approved supplier under GSA in the US and the G-Cloud in the UK, we’ve also passed a Cyber Essentials certification in the UK, and we’re looking to pass other national and international accreditations. In Europe, the GDPR is a focus for European customers, and we’re making sure that we’re complaint with the GDPR. That compliance will help our customers around the world ensure that we follow best practices in data protection. But the main security issues are essentially the same in every country in the world and every geography.

What tools do we offer our customers to help them protect their own data?

The Questionmark software has a strong authentication structure in place—you can define password policies that allow you to have strong passwords. We support integration with SAML, which many of our customers use with their own systems. And then Questionmark has very flexible capabilities to give differential access to data. You can set up your system so only those people who need to know have access to data in your Questionmark system. These are the kinds of features in the product that genuinely increase security.

Why the trust center?

We know with all the concerns about cyber security that there is a real need for our customers to understand whether we as a supplier are secure and safe for them, and so we wanted to create a single place to feature these different accreditations and validations.

There are only a few ways to prove you are secure. Let’s take the example of a castle—you or I could look at this massive structure and guess that if it’s secure or not. But if we’re not castle experts, we really can’t be sure. Similarly, unless you’re a software as a service security expert, how do you know if company A, B or C is secure? By getting to see the different certifications and laws that we follow, customers can get an appreciation of the security we offer, and compare that to others.

What is the future for Questionmark’s commitment to security?

You must have continued improvement. Threats are evolving, and you just can’t stand still. As a very simple example, we currently test our own employees on data security every year, and we’re expanding that by planning different tests by role or department. We’re looking at other accreditations and a continual increase in technical controls. We have many internal improvements that we’re expanding to implement soon. It’s literally a process of continual improvement, and we will continue to add to the trust center.

Questionmark is committed to the highest levels of trust, transparency, and compliance. Please register for a free intro webinar for more information on our key features and functions. We look forward to building your trust and working with you to secure your data.

How do you improve exam integrity and candidate satisfaction?

Posted by Julie Delazyn

Proxy testing … Cheating …. Content theft. How can you be sure that the integrity of your exam isn’t compromised when the technologies and techniques are complex and continually evolving? Many organizations are increasingly turning to online proctoring solutions.

Research shows that when properly applied, online proctoring can be as or more effective than traditional test-center proctoring. Besides reducing security risks, it also improves candidate satisfaction by offering the convenience of taking tests from their homes, workplaces or colleges. It also reduces costs not only for administrators but also for test takers. Consider for a moment the testing fees, travel, parking, and the simple time away from work or studies.  In addition to online proctoring’s effectiveness, one of the most exciting things is that this approach has the potential to grow as technology becomes more advanced and widespread.

What do you need to consider before implementing online proctoring? What features are important to you? How will your exams retain their integrity? How are other colleges or credentialing organizations using online proctoring to improve candidate experience?

Join Questionmark for a 30-minute webinar answering these questions and more

The webinar will cover:

  • Record and review based system with automated flagging
  • What colleges, universities and credentialing organizations are using to offer 24/7 flexibility to test-takers
  • How to administer online exams in a fair way by combining test delivery with exam proctoring

Want to learn more? Secure your spot for this complimentary webinar

Next Page »