New White Paper Examines how to Assess for Situational Judgment

Posted by John Kleeman

Is exercising judgment a critical factor in the competence of the employees and contractors who service your organization? If the answer to this is yes, as it most likely is, you may be interested in Questionmark’s white paper, just published this week on “Assessing for Situational Judgment”.

It’s not just CEOs who need to exercise judgment and make decisions, almost every job requires an element of judgment. Situational Judgment Assessments (SJAs) present a dilemma to the participant and ask them to choose options in response.


Context is defined -> There is a dilemma that needs judgment -> The participant chooses from options -> A score or evaluation is made

Here is an example: 

You work as part of a technical support team that produces work internally for an organization. You have noticed that often work is not performed correctly or a step has been omitted from a procedure. You are aware that some individuals are more at fault than others as they do not make the effort to produce high quality results and they work in a disorganized way. What do you see as the most effective and the least effective responses to this situation?
A.  Explain to your team why these procedures are important and what the consequences are of not performing these correctly.
B.  Try to arrange for your team to observe another team in the organisation who produce high quality work.
C.  Check your own work and that of everyone else in the team to make sure any errors are found.
D.  Suggest that the team tries many different ways to approach their work to see if they can find a method where fewer mistakes are made.

In this example, option C deals with errors but is time consuming and doesn’t address the behavior of team members. Option B is also reasonable but doesn’t deal with the issue immediately and may not address the team’s disorganized approach. Option D is asking a disorganized team to engage in a set of experiments that could increase rather than reduce errors in the work produced. This is likely to be the least effective of the options presented. Option A does require some confidence in dealing with potential pushback from the other team members, but is most likely to have a positive effect.

You can see some more SJA examples at http://www.questionmark.com/go/example-sja.

SJA items assess judgment and variations can be used in pre-hire, post-hire training, for compliance and for certification. SJAs offer assessment programs the opportunity to move beyond assessments of what people know (knowledge of what) to assessments of how that knowledge will be applied in the workplace (knowledge of how).

Questionmark’s white paper is written as a collaboration by Eugene Burke, well known advisor on talent, assessment and analytics and myself. The white paper is aimed at:

  • Psychometricians, testing professionals, work psychologists and consultants who currently create SJAs for workplace use (pre-hire or post-hire) and want to consider using Questionmark technology for such use
  • Trainers, recruiters and compliance managers in corporations and government looking to use SJAs to evaluate personnel
  • High-tech or similar certification organizations looking to add SJAs to increase the performance realism and validity of their exam

The 40 page white paper includes sections on:

  • Why consider assessing for situational judgment
  • What is an SJA?
  • Pre-hire and helping employers and job applicants make better decisions
  • Post-hire and using SJAs in workforce training and development
  • SJAs in certification programs
  • SJAs in support of compliance programs
  • Constructing SJAs
  • Pitfalls to avoid
  • Leveraging technology to maximize the value of SJAs

Situational Judgment Assessments are an effective means of measuring judgment and the white paper provides a rationale and blueprint to make it happen. The white paper is available free (with registration) from https://www.questionmark.com/sja-whitepaper.

I will also be presenting a session about SJAs in March at the Questionmark Conference 2018 in Savannah, Georgia – visit the conference website for more details.

Badging and Assessment: If they know it, let them show it!

Posted by Brian McNamara

We are delighted to announce the availability of Questionmark Badging!

With Questionmark Badging and Questionmark OnDemand, you can grant “badges” to participants based on the outcomes achieved on assessments such as certification exams, post-course tests or advancement exams.Badges associated with Questionmark assessments provide participants with portable, verifiable digital credentials.

Badges aligned with Questionmark assessments can be tied in with competencies and achievements, helping organizations provide recognition and motivation for increasing knowledge and skills. For credentialing and awarding bodies, they can increase the visibility and value of certification programs.

The new app couples Questionmark’s capabilities in delivering valid, reliable and trustworthy assessments with the industry-leading digital credentialing platform from Credly. More than just a visual representation of accomplishment, digital badges provide participants with verifiable, portable credentials that can be shared and displayed across the web, including social networking sites such as LinkedIn, Facebook and Twitter.

Find more info about Questionmark Badging right here!

Questionmark Conference 2018 – Registration is Open!

Posted by Brian McNamara

We are excited to announce that registration for Questionmark Conference 2018 is now open!

Questionmark customers will gather March 6th – 9th in Savannah, Georgia, for three days of learning, networking and professional development opportunities.   Whether you are a new user or a pro, Questionmark Conference is the ideal place to get vital information, insight and learning about the latest assessment technologies.

Join us at the Westin Savannah Harbor to enjoy southern hospitality with your peers and the Questionmark team to:

  • Learn about new Questionmark features and functions
  • Attend engaging hands-on product training sessions
  • Network with fellow Questionmark users
  • Get valuable tips, insights and best-practice guidance
  • Plan for the future with a sneak peek at the product road map

Registration is now open at special early-bird rates. Visit the conference website today for details!

Questionmark Conference 2018: Assess for Success | March 6-9 in Savannah

Questionmark is ISO 27001 certified. What this means for you?

Posted by John Kleeman

As you may have seen on our news site, Questionmark has just been certified to the Information Security standard ISO 27001.

What is ISO 27001?Chart showing probability and impact of risks, with high impact and high probability risks in red

ISO 27001 (full name ISO/IEC 27001:2013) is an international information security standard that is widely recognized as credible and authentic in validating that the certified organization has an effective management system for security.

The core of ISO 27001 is risk management. You identify in a systematic way risks to confidentiality, integrity, and availability and then assess their impact and probability. As simplistically shown in the diagram to the right, you decide what risks you can accept and how you can mitigate or otherwise deal with those that you cannot accept.

Subject to risk assessment, ISO 27001 requires you to meet over 100 controls including all the usually expected security controls. It also requires top management commitment and very specific processes to deal with issues that arise and auditing and much more.

ISO 27001 also encourages continual improvement – with all the threats out there, you have to keep making your processes and security better.

Questionmark’s ISO 27001 journey

Security has been central to Questionmark’s mission for decades. We brought out the world’s first secure browser in the 1990s, and our very first post in this blog back in 2009 was about delivering assessments safely and securely.

Last year, we decided that if we were to get external audit and validation of our security, it would both help us become more secure and help customers and other stakeholders feel more comfortable with our service. We’d been aware of 27001 for some time as the most credible security standard out there, and decided to adapt our processes and internal documentation to meet it. And we commissioned BSI, who are leaders in this field, to audit us.

The process to become certified is quite arduous. Including “internal” audits by a consultant and BSI’s audits, we have had eight days of auditing in the last few months. And these can be quite grueling – one of our audit days started with breakfast at 7 am and the auditor left the building just after 7.30pm at night! This definitely puts your people, processes, and technology through their paces. Implementing 27001 has improved Questionmark security and I’d encourage you to respect any organization who is certified as it’s a very credible process.BSI Assurance Mark Template RGB

I’m pleased to let you know that we are now certified by BSI under ISO 27001. Our certificate number is IS 668255. Our scope and certificate of applicability are wide, and we’d be pleased to share these with stakeholders under NDA.

How might it matter to purchasers of assessment services?

ISO 27001 certification gives external validation that an organization has a good quality information security management system.

Anyone can claim to be secure. Anyone can claim to follow standards. It’s hard for someone who is not a security expert to know whether an organization actually has put the effort into people, process, and technology to do the best that can be done to resist threats to confidentiality, integrity and availability.

With all the threats out there to assessment data, we believe it’s helpful to our customers to have assurance that Questionmark has been independently audited and it has been certified that our information security management system complies with ISO 27001.

How could ISO 27001 help assessment providers?

Are you looking to create and deliver secure assessments and keen to protect confidentiality, integrity, and availability?

Although using Questionmark OnDemand will help you do this, I’d encourage some blog readers to think whether it might make sense to implement ISO 27001 yourselves as an organization. That way you will ensure that all your IT and systems are securely managed. There are some work and effort involved, but it will make you as an organization more secure and less likely to suffer breaches and other failures.

I’ve just led Questionmark’s implementation of ISO 27001 and would be happy to share experiences with others in the assessment industry, please feel free to reach out to me.

Questionmark’s assessment management system now offered in the G-Cloud digital marketplace

Posted by Chloe Mendonca

We’re excited to announce that Questionmark has been accepted as a G-Cloud 9 supplier of assessment solutions by the Crown Commercial Service (CCS). These solutions can be found on the UK government’s Digital Marketplace.

What is G-Cloud?

The UK government G-Cloud streamlines the process by which public-sector bodies procure cloud-based applications. You can think of G-Cloud like a mobile app store that contains a huge range of approved, ready-to-use services and applications.

What does this mean for public-sector organisations?
If you are a UK public-sector organisation, G-Cloud makes it easier to buy Questionmark’s assessment management system by dramatically reducing the time you spend procuring services — quickly connecting you with a supplier that fits your strict requirements and budget. US public-sector organisations can procure Questionmark via a similar service, the US General Services Administration (GSA) federal supply list. Organisations outside the US and UK can take comfort in the fact that Questionmark’s platform has been vetted and approved for listing on these government service provider sites.

Questionmark has excelled for many years in delivering assessment solutions to the public sector. Being available on G-Cloud enables Questionmark to expand our offering to even more government organisations that need to create, deliver and report on tests, exams, quizzes and surveys.

Why Questionmark?
In today’s highly-regulated world, the need to effectively assess knowledge and understanding of regulations or corporate policies is critical. Organisations are now required to do more than the traditional checkbox compliance approach and must ensure that all employees or target groups understand the rules and can follow them. Failure to do so can impact life, limb and livelihood, often resulting in fines and damaged business reputations. Only by using secure assessment management technologies that seamlessly integrate with your other enterprise systems and learning management tools can organisations efficiently and effectively set up, deploy and monitor compliance.

If you’re in a public-sector organisation looking for an intuitive, scalable SaaS solution that will help you ensure regulatory compliance and measure learning, request a demo of Questionmark’s assessment management system today.

Questionmark OnDemand Assessment Management System now HIPAA-compliant

Jamie ArmstrongPosted by Jamie Armstrong

Questionmark recently began offering US OnDemand Service customers the option of entering into an additional agreement for compliance with HIPAA (the US Health Insurance Portability and Accountability Act).

I’d like to provide some brief information on this exciting new development, particularly for those not familiar with what HIPAA is or involves. You can easily find additional information and resources on the U.S. Department of Health & Human Services website.

What is HIPAA and what kind of information or data does it cover?
HIPAA is a US federal law that in very general terms regulates access to and handling of “protected health information” (“PHI”) and provides individuals with important rights regarding their health information. PHI includes these categories of information:

  • health information collected from a person;
  • information relating to health conditions or health care provision created or received by an organization such as a health care provider, and;
  • information that either identifies or can reasonably be used to identify an individual.

For example, data gathered or used as part of an assessment using Questionmark OnDemand that relates to past, present or future health or condition may be PHI under HIPAA.

What types of organizations are subject to HIPAA requirements?
HIPAA applies to two main categories of organization having access to PHI. These are known as “covered entities” and “business associates.” A Questionmark customer that is a health plan or health care provider, e.g. a hospital, clinic or health insurance company,  may be a covered entity for HIPAA. Business associates include organizations receiving or maintaining PHI on behalf of a covered entity for functions such as data processing or administration (among other things). Questionmark may be a business associate in providing the OnDemand Service to customers that are either covered entities or business associates performing services for their own covered-entity clients.

What does HIPAA require?
HIPAA requires that covered entities and business associates meet various security, breach notification and privacy requirements. They must meet the requirements applicable to them internally and also have contracts with any third parties that may have access to PHI. This ensures  that these third parties are subject to the same restrictions and conditions. Before offering OnDemand Service customers the option of entering into a HIPAA business associate agreement, Questionmark completed a security and legal review to ensure compliance with relevant HIPAA requirements.

We are interested in obtaining HIPAA-compliant OnDemand Services.  How do we sign a HIPAA business associate agreement with Questionmark?
You can find our HIPAA business associate agreement here. If you’d like to learn more please contact your account manager. Questionmark is committed to safeguarding PHI in accordance with the HIPAA standards and looks forward to discussing your HIPAA compliance requirements.

Important disclaimer: This blog is provided for general information and interest purposes only, is non-exhaustive and does not constitute legal advice. As such, the contents of this blog should not be relied on for any particular purpose and you should seek the advice of your own legal counsel in considering HIPAA requirements.