SAML 101

Bart Hendrickx SmallPosted by Bart Hendrickx

As I mentioned in my previous post on SSO, Single Sign-On: Who’s Involved?, we’ll take a look at SAML to understand what it is and how it’s used with SSO. In this post I’ll explain what SAML is, and I will offer an example use case in my next post.

Webinar

So, What Is SAML?

SAML, or Security Assertion Markup Language, is a protocol that allows systems to exchange authentication data on users. (It facilitates other use cases as well, but I will focus on authentication.) What does that mean? It means that one system can ask: “Who is this user?” and another system can answer: “This is Jane Doe.” As I mentioned in my post previous post, I am talking about the service provider (SP) and identity provider (IdP) respectively.

Service providers (SP) in this context can be any software system with which you can do something, such as sending and receiving email, tracking projects or delivering assessments. Similarly, an identity provider (IdP) can be any software system that contain data on users that you can use to determine who those users are.

If you manage an SP, you probably don’t want just any IdP telling you who someone is. You will typically trust only one or a few IdPs. And if you are in charge of an IdP, you will likewise prefer to send user data only to those SPs you know and trust. To accomplish that, the SP and IdP exchange data allowing them to establishing a trust relationship. Those data are often called federation metadata, federation referring to the fact that there is an alliance between the different systems.

SAML is a popular protocol to set up such federations between service providers and identity providers. Look up SAML in your favorite search engine and you will get many results. One of its advantages is that it is extensible, meaning that you can exchange information that is relevant to your situation. For example, do you have an IdP that stores the hire date for an employee (or enrollment date of a student)? Do you want to share those data with an SP so that it can decide whether the user is allowed to access a certain resource? Then you can set up the federation in such a way that the IdP will send an attribute for hire (or enrollment) date to the SP.

Another advantage, and it is a huge one, is that SAML can be used in situations where the IdP and SP cannot talk to each other, for example because they are on different networks. You may have an IdP running on your internal network, behind a firewall. Your SP may be available in the cloud, as is the case with Questionmark OnDemand. The SP cannot talk to the IdP because it cannot “see” it. However, that’s not a problem for SAML. In my next post, we’ll take a look at a typical use case so we can see the practicality of using SAML with SSO.

 

Online Proctoring: FAQs

John Kleeman HeadshotPosted by John Kleeman

Online proctoring was a hot-button topic at Questionmark’s annual Users Conference. And though we’ve discussed the pros and cons in this blog and even offered an infographic highlighting online versus test-center proctoring, many interesting questions arose during the Ensuring Exam Integrity with Online Proctoring  session I presented with Steve Lay at Questionmark Conference 2016.

I’ve compiled a few of those questions and offered answers to them. For context and additional information, make sure to check out a shortened version of our presentation. If you have any questions you’d like to add to the list, comment below!

What control does the online proctor have on the exam?

With Questionmark solutions, the online proctor can:

  • Converse with the participant
  • Pause and resume the exam
  • Give extra time if needed
  • Terminate the exam

What does an online proctor do if he/she suspects cheating?

Usually the proctor will terminate the exam and file a report to the exam sponsor.

What happens if the exam is interrupted, e.g. by someone coming in to the room?

This depends on your security protocols. Some organizations may decide  to terminate the exam and require another attempt. In some cases, if it seems an honest mistake, the organization may decide that the proctor can use discretion to permit the exam to continue.

Which is more secure, online or face-to-face proctoring?online proctoring

On balance, they are about equally secure.

Unfortunately there has been a lot of corruption with face-to-face proctoring, and online proctoring makes it much harder for participant and proctor to collude as there is no direct contact, and all communication can be logged.

But if the proctors are honest, it is easier to detect cheating aids in a face-to-face environment than via a video link.

What kind of exams is online proctoring good for?

Online proctoring works well for exams where:

  • The stakes are high and so you need the security of a proctor
  • Participants are in many different places, making travel to test centers costly
  • Participants are computer literate – have and know how to use their own PCs
  • Exams take 2-3 hours or less

If your technology or subject area changes frequently, then online proctoring is particularly good because you can easily give more frequent exams, without requiring candidates to travel.

What kind of exams is online proctoring less good for?

Online proctoring is less appropriate for exams where:

  • Exams are long and participants needs breaks
  • Exams where participants are local and it’s easy to get them into one place to take the exam
  • Participants do not have access to their own PC and/or are not computer literate

How do you prepare for online proctoring?

Here are some preparation tasks:

  • Brief and communicate with your participants about online proctoring
  • Define clearly the computer requirements for participants
  • Agree what happens in the event of incidents – e.g. suspected cheating, exam interruptions
  • Agree what ID is acceptable for participants and whether ID information is going to be stored
  • Make a candidate agreement or honor code which sets out what you expect from people to encourage them to take the exam fairly

I hope these Q&A and the linked presentation are interesting. You can find out more about Questionmark’s online proctoring solution here.

Job Task Analysis Surveys Legally Required?

John Kleeman Headshot

Posted by John Kleeman

I had a lot of positive feedback on my blog post Making your Assessment Valid: 5 Tips from Miami. There is a lot of interest in how to ensure your assessment is valid, ensuring that it measures what it is supposed to measure.

If you are assessing for competence in a job role or for promotion into a job role, one critical step in making your assessment valid is to have a good, current analysis of what knowledge, skills and abilities are needed to do the job role. This is called a job task analysis (JTA), and the most common way of doing this analysis is to conduct a JTA Survey.

Job Task Analysis SurveyIn a JTA Survey, you ask existing people in the job role, or other experts, what tasks they do. A common practice is to survey them on how important each task is, how difficult it is and how often it is done. The resultant reports then guide the construction of the test blueprint and which topics and how many questions on each you include in the test.

If you cannot show that your assessment matches the requirements of a job, then your assessment is not only invalid but it is likely unfair — if you use it to select people for the job or measure competence in the job. And if you use an invalid assessment to select people for promotion or recruitment into the job, you may face legal action from people you reject.

Not only is this common sense, but it was also confirmed by a recent US district court ruling against the Boston Police Department. In this court case, sergeants who had been rejected for promotion to lieutenant following an exam sued that the assessment was unfair, and won.

The judge ruled that the exam was not sufficiently valid, because it omitted many job skills crucial for a police lieutenant role, and so it was not fair to be used to select for the role (see news report).

The 82-page judge’s ruling sets out in detail why the exam was unfair. He references the Uniform Guidelines on Employee Selection Procedures which state:

“There should be a job analysis which includes an analysis of the important work behavior(s) required for successful performance and their relative importance”

But the judge ruled that although a job analysis had been done, it had not been used properly in the test construction process. He said:

“When using a multiple choice exam, the developer must convert the job analysis result into a test plan to ensure a direct and strong relationship between the job analysis and the exam.

However, in this case, the job analysis was not used sufficiently well to construct the exam. The judge went on to say:

The Court cannot find, however, that the test plan ensured a strong relationship between the job analysis and the exam. … too many skills and abilities were missing from the … test outline. 

Crucially, he concluded:

“And a high score on the … exam simply was not a good indicator that a candidate would be a good lieutenant”.

Due to the pace of business change and technological advance, job roles are changing fast. Make sure that you conduct regular JTAs  of roles in your organization and make sure your assessments match the most important job tasks. Find out more about Job Task Analysis here.

Making your Assessment Valid: 5 Tips from Miami

John Kleeman Headshot

Posted by John Kleeman

A key reason people use Questionmark’s assessment management system is that it helps you make more valid assessments. To remind you, a valid assessment is one that genuinely measures what it is supposed to measure. Having an effective process to ensure your assessments are valid, reliable and trustable was an important topic at Questionmark Conference 2016 in Miami last week. Here is some advice I heard:

Reporting back from 3 days of learning and networking at Questionmark Conference 2016 in Miami

Tip 1: Everything starts from the purpose of your assessment. Define this clearly and document it well. A purpose that is not well defined or that does not align with the needs of your organization will result in a poor test. It is useful to have a formal process to kick off  a new assessment to ensure the purpose is defined clearly and is aligned with business needs.

Tip 2: A Job Task Analysis survey is a great way of defining the topics/objectives for new-hire training assessments. One presenter at the conference sent out a survey to the top performing 50 percent of employees in a job role and asked questions on a series of potential job tasks. For each job task, he asked how difficult it is (complexity), how important it is (priority) and how often it is done (frequency). He then used the survey results to define the structure of knowledge assessments for new hires to ensure they aligned with needed job skills.

Tip 3: The best way to ensure that a workplace assessment starts and remains valid is continual involvement with Subject Matter Experts (SMEs). They help you ensure that the content of the assessment matches the content needed for the job and ensure this stays the case as the job changes. It’s worth investing in training your SMEs in item writing and item review. Foster a collaborative environment and build their confidence.

Tip 4: Allow your participants (test-takers) to feed back into the process. This will give you useful feedback to improve the questions and the validity of the assessment. It’s also an important part of being transparent and open in your assessment programme, which is useful because people are less likely to cheat if they feel that the process is well-intentioned. They are also less likely to complain about the results being unfair. For example it’s useful to write an internal blog explaining why and how you create the assessments and encourage feedback.

Lunch with a view at Questionmark Conference 2016 in Miami

Tip 5: As the item bank grows and as your assessment programme becomes more successful, make sure to manage the item bank and review items. Retire items that are no longer relevant or when they have been overexposed. This keeps the item bank useful, accurate and valid.

There was lots more at the conference – excitement that Questionmark NextGen authoring is finally here, a live demo of our new easy to use Printing and Scanning solution … and having lunch on the hotel terrace in the beautiful Miami spring sunshine – with Questionmark branded sunglasses to keep cool.

There was a lot of buzz at the conference about documenting your assessment decisions and making sure your assessments validly measure job competence. There is increasing understanding that assessment is a process not a project, and also that to be used to measure competence or to select for a job role, an assessment must cover all important job tasks.

I hope these tips on making assessments valid are helpful. Click here for more information on Questionmark’s assessment management system.

Single sign-on: secure and easy access

Bart Hendrickx Small

Posted by Bart Hendrickx

It’s Tuesday morning. You have just started your computer. Now it’s time to open your day-to-day tools: email, chat/phone, tasks and so on. As you go through your tasks, you realize you need to take that data security test you’ve been postponing.

Every day, you interact with various applications. Some are installed on your personal computer, others on servers managed either by your organization by vendors.  Your applications might come from a multitude of service providers, in-house or on the Cloud.

For many of those applications, you need to authenticate—to tell the applications who you are—so that the app can present the information that pertains to you. Sometimes this happens automatically. When your email client connects to the mail server, you read your emails, not those of your co-workers. Your email client has authenticated you against your mail server because you entered a username or email address, a password and some other data, way back when.

You often need to use different sign-ins for different apps. When you log in tor you Questionmark OnDemand portal, for instance, you enter a different username and password than the one you used to unlock your computer earlier today, (Your organization’s data security policy does not allow you to  store your organizational password in other systems.)

Want to learn more? I’ll be discussing this topic and more at the Questionmark Conference 2016 in Miami, April 12-15. Register before March 3 to take advantage of our final early-bird discounts.

Problem: Unrecognized username or password

You’ve logged in for your exam, but you get an error message. Maybe you mistyped the password? Second try. Nope; same results. You must have forgotten your password. You start an instant message window with your internal IT help desk. “Sorry, we don’t manage Questionmark OnDemand. Can you use its password reset function?”

You go back to your Questionmark login page, get a secure on-time login and establish a new, permanent password that complies with the data security policy — “I better not forget my password this time,” you say to yourself as you finally start your data security test. “Isn’t there something more convenient?”

Solution: Single sign-on

We all find ourselves in similar situations, but with Single sign-on (SSO) we can avoid them.

Since, there are several definitions of SSO, here’s how I’ll define it in the context of this blog:

Single Sign-On (SSO) for software is the ability for one application, the identity provider, to tell another application, the service provider, who you are.

By identity provider, I mean a system that contains digital identity information—also known as people data—on users, For example, think of social network sites or Active Directory from Microsoft.

The service provider is the system that users work with to do something—say Questionmark OnDemand, in the case of your data security test.

With SSO, a user does not log on directly to the service provider. Instead, they log on to an identity provider, which then tells the service provider who the user is. The identity provider and service provider have been configured to trust each other. So when the identity provider says: “This is Jane Doe,” the service provider will trust and accept that.

It is important to note that SSO is therefore not about creating accounts with the same usernames and passwords—a prevalent mechanism for different service providers. SSO is about making those service providers accept what an identity provider says about a user.

Why SSO?

SSO comes with several advantages. Users can access all applications that are linked to their identity providers—using one username and password for multiple systems. Depending on the capabilities of the applications and how things have been set up, the authentication can be seamless. You might log on to your identity provider when you start your computer, and the other applications (service providers) you access during the day will automatically check with your identity provider without you having to enter your username and password again.

SSO makes password management easier for IT administrators. Having an employee leave an organization might mean having to decommission access to dozens of service providers. If the authentication to those service providers has been set up with SSO, then an IT administrator only needs to decommission the employee’s identity provider account. Without that account, the employee can no longer log on to any of the linked applications.

There is one disadvantage to SSO: If the account at the identity provider is hacked, all linked applications can be compromised. It is therefore imperative the account is properly secured. How can you set up SSO to ensure its security and effectiveness? Watch for more posts on this subject, which will include information about our newly added support for a popular technique called SAML.

If you would like to learn more, attend my session: Secure Authentication: Accessing Questionmark OnDemand with SSO at the Questionmark Conference 2016, April 12-15. Register before March 3 to take advantage of our final early-bird discounts.

9 trends in compliance learning, training and assessment

John Kleeman HeadshotThis version is a re-post of a popular blog by John Kleeman

Where is the world of compliance training, learning and assessment going?

I’ve collaborated recently with two SAP experts, Thomas Jenewein of SAP and Simone Buchwald of EPI-USE, to write a white paper on “How to do it right – Learning, Training and Assessments in Regulatory Compliance[Free with registration]. In it, we suggested 9 key trends in the area. Here is a summary of the trends we see:

1. Increasing interest in predictive or forward-looking measures

Many compliance measures (for example, results of internal audits or training completion rates) are backwards looking. They tell you what happened in the past but don’t tell you about the problems to come. Companies can see clearly what is in their rear-view mirror, but the picture ahead of them is rainy and unclear. There are a lot of ways to use learning and assessment data to predict and look forward, and this is a key way to add business value.

2. Monitoring employee compliance with policies

A recent survey of chief compliance officers suggested that their biggest operational issue is monitoring employee compliance with policies, with over half of organizations raising this as a concern. An increasing focus for many companies is going to be how they can use training and assessments to check understanding of policies and to monitor compliance.

3. Increasing use of observational assessments

Picture of observational assessment on smartphoneWe expect growing use of observational assessments to help confirm that employees are following policies and procedures and to help assess practical skills. Readers of this blog will no doubt be familiar with the concept. If not, see Observational Assessments—why and how.

4. Compliance training conducted on mobile devices

The world is moving to mobile devices and this of course includes compliance training and assessment.

5. Informal learning

You would be surprised not to see informal learning in our list of trends. Increasingly we are all understanding that formal learning is the tip of the iceberg and that most learning is informal and often on the job.

6. Learning in the extended enterprise

Organizations are becoming more interlinked, and another important trend is the expansion of learning to the extended enterprise, such as contractors or partners. Whether for data security, product knowledge, anti-bribery or a host of other regulatory compliance reasons, it’s becoming crucial to be able to deliver learning and to assess not only your employees but those of other organizations who work closely with you.

7. Cloud

There is a steady movement towards the cloud and SaaS for compliance learning, training, and assessment – with the huge advantage of delegating all of the IT to an outside party being the strongest compelling factor.  Especially for compliance functions, the cloud offers a very flexible way to manage learning and assessment without requiring complex integrations or alignments with a company’s training departments or related functions.

8. Changing workforce needs

The workforce is constantly changing, and many “digital natives” are now joining organizations. To meet the needs of such workers, we’re increasingly seeing “gamification” in compliance training to help motivate and connect with employees. And the entire workforce is now accustomed to seeing high-quality user interfaces in consumer Web sites and expects the same in their corporate systems.

9. Big Data

E-learning and assessments are a unique way of touching all your employees. There is huge potential in using analytics based on learning and assessment data. We have the potential to combine Big Data available from valid and reliable learning assessments with data from finance, sales, and HR sources.  See for example the illustration below from SAP BusinessObjects showing assessment data graphed against performance data as an illustration of what can be done.

data exported using OData from Questionmark into SAP BusinessObjects

For information on these trends, see the white paper written with SAP and EPI-USE: “How to do it right – Learning, Training and Assessments in Regulatory Compliance”, available free to download with registration.

If you have other suggestions for trends, feel free to contribute them below.

« Previous PageNext Page »