FBI and Homeland Security advice on trumping cybersecurity attacks

Posted by John Kleeman

There’s a lot in the news recently about possible cybersecurity attacks on the political process. Here are some thoughts on how we can learn from this and apply it to assessment security.

One of the most interesting documents I’ve read on this subject is the Department of Homeland Security and FBI’s joint analysis report  JAR-16-20296 titled GRIZZLY STEPPE – Russian Malicious Cyber Activity.  This presents evidence on how a cybersecurity attack was made on a US political party in 2016 and gives some practical advice on how others can set up their systems to avoid such attacks.

Whoever the attack was performed by (and there has been some debate about this), the practical advice is useful to anyone who wants to improve their security. I was particularly struck by a section in the report which offered questions to ask your organization to see if they have good cybersecurity practices. I’ve taken the liberty of including the questions in the graphic below:

See Grizzly Steppe report for text here

I’ve shared various sets of security questions in this blog, including Eight ways to check if security is more than skin deep and 24 midsummer questions to ask your assessment software provider, but here are some questions from a very credible source!

I’d encourage you to pose these questions within your organization and with your suppliers to check that you are well protected in case of a cyberattack. Questionmark, like all sensible organizations, believes in continuous improvement in our security, and listening to sources like this analysis informs our improvement.

I hope highlighting the report and these questions helps strengthen your defenses against cybersecurity and acts as a guide in choosing your vendors.

Seven New Year’s Resolutions to Keep Your Assessments Safe

Paper with "Resolutions" written on it implying one is about to write some resolutions downJohn Kleeman HeadshotPosted by John Kleeman

Many blogs at this time of year seek to predict the year ahead, and many of them foresee more data breaches and security incidents in 2017.  But I’m a great believer that the best way to predict the future is to create or change it yourself. So if you want to reduce the chances of your assessment data security being breached in 2017, make some of the things you’ve talked about happen.

Here are some possible New Year’s resolutions that could help keep your assessments safe and secure.

1. Audit your user accounts. Go through each of your systems that hold or give access to assessment data, and check there are no accounts for ex-employees or ex-contractors. Make sure there are no generic or test accounts that do not belong to a current individual. Dormant accounts like this are a common route to a breach. Also check that no one who has changed role has the privileges of their old role.

2. Run an incident response table-top practice exercise. This is a session where you gather together those responsible for security, pretend there is a breach or other incident and work through verbally how you’d deal with it as a team. You can do this in a couple of hours with good preparation, and it allows you to check your procedures and ensure people know what to do. It will often give useful insight into improving your preparedness.  As Benjamin Franklin once said “An ounce of prevention is worth a pound of cure”.

3. Start testing your personnel on security procedures. One of the biggest security risks for any organization is staff mistakes and accidents that compromise credentials or data. Security awareness training makes an important difference. And if you test your personnel on security after the training, you verify that people understand the training and you identify areas of weakness. This makes it more likely that your personnel become more aware and follow better security practices. If you have access to an online assessment tool like Questionmark, it’s very, very easy to do.

Photo of doctor stethoscope on computer keyboard4. Review some of your key vendors. A risk for most organizations is weaknesses in suppliers or subcontractors that have access to your data. Ask suppliers to share information on their technical and organizational measures for security and what they are doing to ensure that your data is not breached. Any reputable organization will be willing and able to provide this under NDA. See 24 midsummer questions to ask your assessment software provider on this blog for some of the questions you can ask.


5. Conduct a restore test from backups. How do you know your backups work? Over the years, I’ve come across a few organizations and teams who’ve lost their data because their backups didn’t work. The only way to be sure is to test restoring it from backup and check data is there. If you don’t already run restore tests, organize a restore test in 2017 (ideally once a quarter, but once is better than not at all). You shouldn’t need to do this if you use a cloud service like Questionmark OnDemand as the vendor should do it for you.

6. Run a pilot for online proctoring. Microsoft do it. SAP do it. Why shouldn’t you do it? If you run a certification program that uses physical test centers, consider whether online proctoring might work for you. Not only will it reduce the risk of collusion with proctors helping candidates cheat, but it will also be a huge boon to your candidates who will no longer need to travel to test centers.

TheCadetHonorCodeMonument7. Put in place a code of conduct for your participants. This is a simple thing to do and can make a big difference in reducing cheating by encouraging test-takers to stay honest.  See Candidate Agreements: Establishing honor codes for test takers and What is the best way to reduce cheating? on this blog for tips on how and why to do this. If you are looking for inspiration, at famous code of conduct is that of the U.S. Army West Point Military Academy which simply says: “A cadet will not lie, cheat, steal, or tolerate those who do.” Of course you need to communicate and get buy-in for your code of conduct, but if you do, it can be very effective.

Many of you will already be doing all of these things, but if you’re not, I hope one or more of these resolutions help you improve your assessment security in 2017.

And here’s a bonus New Year’s resolution to consider. Questionmark Information Security Officer David Hunt and I are giving a session on Staying Ahead of Evolving Security Threats at the Questionmark conference in March in Santa Fe. Make a New Year’s resolution to come to the conference, and learn about security and assessment!

Assessment Security: 5 Tips from Napa

John Kleeman Headshot

Posted by John Kleeman

Assessment security has been an important topic at Questionmark, and that was echoed at the Questionmark Users Conference in Napa last week. Here is some advice I heard from attendees:

  •  Tip 1: It’s common to include an agreement page at the start of the assessment, where the participant agrees to follow the assessment rules, to keep exam content confidential and not to cheat. This discourages cheating by reducing people’s ability to rationalize that it’s okay to do so and also removes the potential for someone to claim they didn’t know the rules.
  • Tip 2: It’s a good idea to have a formal agreement with SMEs in your organization who author or review questions to remind them not to pass the questions to others. If they are employees, you should involve your HR and legal departments in drafting the agreements or notices. That way if someone leaks content, you have HR and legal on board to deal with the disciplinary consequences.Data gathering, screening, unproctored assessments, proctored assessments
  • Tip 3: It’s prudent to use the capabilities of Questionmark software to restrict access to the item bank by topic. Only give  authors access to the parts they are working on, to avoid inadvertent or deliberate content leakage.
  • Tip 4: There is increasing interest and practical application of hybrid testing strategies for proctored and unprotected tests to allow you to focus anti-cheating resources on risk. For example, you might screen participants with quizzes, then give un-proctored tests and give those who pass a proctored test.  Or you might deliver a series of short exams, at periodic intervals to make it harder for people to get proxy test takers to impersonate them. There is also a lot of interest in online proctoring, where people can take exams at home or in the office, and be proctored by a remote proctor using video monitoring.  This reduces travel time and is often more secure than face-to-face proctoring.
  • Tip 5: If your assessment system is on premise (behind the firewall), check regularly with your IT department that they are providing the security you need and that they are backing up your data. Most internal IT departments are hugely competent, but there is a risk as people change jobs over time that your IT department might lose touch with what the assessment application is used for. One user shared how their IT system failed to make backups of the Questionmark database, so when the server failed, they lost their data and had to restart from scratch. I’m sure this particular issue won’t happen for others, but IT teams have a wide set of priorities, so it’s good to check in with them.

There was lots more at the conference – iPads becoming mainstream for authoring and administration as well as delivery, people using OData to get access to Questionmark data, Questionmark being used to test the knowledge of soccer referees and some good thinking on balancing questions at higher cognitive levels.

One thing that particularly interested me was anecdotal evidence that having an internal employee certification program reduces employee attrition. Employees are less likely to leave your organization if you have an assessment and certification program. Certification makes employees feel more valued and more satisfied and so less likely to leave for a new job elsewhere. A couple of attendees shared that their internal statistics showed this.

This mirrors external research I’ve seen – for example the Aberdeen Group have published research which suggests that best-in-class organizations use assessments around twice as often as laggard organizations, and that first-year retention for best-of-class organizations is around 89% vs 76% for laggards.

For more information on security,  download the white paper: Delivering Assessments Safely and Securely.


What is the best way to reduce cheating?

John Kleeman HeadshotPosted by John Kleeman

There is a famous saying: “If you want to build a ship, don’t drum up the people to gather wood, divide the work, and give orders. Instead, teach them to yearn for the vast and endless sea.” This has a useful analogy in preventing cheating.

There are many useful technical and procedural ways of preventing cheating in tests and exams, and these are important to follow, but an additional, cost-effective way of reducing cheating is to encourage participants to choose not to cheat. If you can make your participants want to take the test fairly and honestly — by reducing their rationalization to cheat — this will reduce cheating.

Fraud triangle - Motivation, Opportunity and RationalizationAs shared by my colleague Eric Shepherd  in his excellent blog article Assessment Security and How To Reduce Fraud, cheating at a test is a variant of fraud.  Donald Cressey, a famed criminologist came up with the fraud triangle shown in the diagram to the right to explain why people commit fraud.

In order for someone to commit fraud (e.g. cheat at a test), he or she must have Motivation, Opportunity and Rationalization.  Motivation comes from the stakes of the test; for an important test, this is difficult to reduce. Opportunity arises out of technical and procedural weaknesses in the test-taking process, and you can obviously strengthen processes to reduce opportunity in many ways.

Rationalization is when someone reconciles their bad deeds as acceptable behavior. We all have values and like to think that what we are doing is right. When someone conducts fraud, they typically rationalize to themselves that what they are doing is right or at least acceptable. For example, they convince themselves that the organization they are robbing deserves it or can afford the loss. When cheating at a test, they say to themselves that the test is not fair or that they are just copying everyone else or they find some other excuse to rationalize and feel good about the cheating.

Here are some ways to make it less likely that people will rationalize about cheating on your test:

1. Formalize a code of conduct (e.g. honesty code) which sets out what you expect from test takers. Communicate this effectively well in advance and get people to sign up to it right before taking the test. For example, you can put it on the first screen after they log in. This will reduce rationalization from people who might claim to themselves they didn’t know it was wrong to cheat or that everyone cheats.

2. Provide fair and accessible learning environments where people can learn to pass the assessment honestly, and provide practice exams so people can check their learning. Rationalization is increased if people think there is no other way to pass the test than cheating.

3. Make sure that the test is trustable (reliable and valid) and fair. If the test is not seen as fair,  people will be less like to rationalize that it’s permissible to cheat.

3. Communicate details of why the tests are there, how the questions are constructed and what measures you take to make the Cheat sheet in a juice box test fair, valid and reliable. Again, if people know the test is there for good reason and fair, they will be less motivated to cheat.

4. Maintain a positive public image. This will reduce rationalization by people claiming that  the assessment provider is incompetent or has other faults.

5. Communicate your security measures and how people who cheat are caught.  This makes people less likely to think they will be able to get away with it.

For many organizations — in addition to other anti-cheating measures — it can be very productive to spend time reducing participants’ rationalization to cheat, thereby helping them choose to be honest. The picture on the right shows a “cheat sheet” or “crib sheet” hidden in a juice carton. Think of ways you can encourage participants to use their inventiveness to learn to pass the exam, not to believe it’s okay to defraud you and the testing system.

I hope you find this good practice tip helpful. I’ll be presenting at the Questionmark Users Conference March 10 – 13 on Twenty Testing Tips: Good practice in using assessments. Taking measures to reduce rationalization for cheating will be one of my tips. Register for the conference if you’re interested in hearing more.

Trusting you have a good new year

John Kleeman HeadshotPosted by John Kleeman

As we bid 2014 goodbye  and welcome 2015, we wish all readers of this blog a happy and prosperous new year.

Trust is in the news a lot these days. 2013 was memorable for its revelations of government surveillance of the Internet. Well-intentioned government organizations were intercepting Internet communications for law and order purposes, and to protect society from harm. However the surprising scale of the interceptions divided the community with some feeling it was appropriate given the threat but others becoming less trustful of government.

In 2014, we have seen a series of Internet vulnerabilities. The catchy names – Heartbleed, Shellshock and Poodle – bely the potential seriousness of these threats. Questionmark was only lightly touched by these vulnerabilities and any minor issues were quickly corrected (see Questionmark and Heartbleed and Questionmark not impacted by Bash/ShellShock Internet vulnerability). However as we’ve seen in the news, some other companies have been impacted by these or other vulnerabilities, and we are all very sensibly being more cautious about security and data protection.

Questionmark has and continues to put a high priority on security and data protection. Watch this video for more about Questionmark’s commitment to security.

2014 seems to have been the year that security and data protection have come of age. Mature organizations recognize that there are significant security threats to their data, and mature suppliers put in place extensive measures to protect against such threats. The arguments in favour of outsourcing to the Cloud remain strong; if nothing else, Cloud providers can typically protect specialist data like assessments better than a busy in-company IT team, who are focused elsewhere. But trust must be at  the forefront – you need to trust and review all your suppliers, to check that they are following good security practice. We welcome all the review we get from our customers’ IT security departments   – good questions help make us stronger.

Trust and trustable assessment results are critical to Questionmark. Our vision is that in today’s world, success for organizations, individuals and society means having the right knowledge, skills and abilities at the right place and the right time. An organization needs to know what its people understand and what they need to change or learn to meet goals. An individual needs to demonstrate achievement and find out how to improve. And society needs to know who is competent and whom to trust.

Assessments are critically needed to identify if people “know it, understand it and can do it”. Questionmark aims to provide the world’s leading online assessment service, allowing organizations to securely create, deliver and report on tests, quizzes, surveys and exams. Questionmark focuses on getting trustable results that are actionable for organizations, individuals and society.

During 2015 we’ll be sharing lots about assessment and good practice on this blog, and I trust we will have much to interest you!


Assessment Security: Reducing Fraud

julie-smallPosted by Julie Chazyn

I’d like to draw your attention to some recent writing by our CEO, Eric Shepherd, on the subject of “Assessment Security and How To Reduce Fraud.”fraud risk image

In this post on his blog, Eric describes  what motivates fraudulent behavior and explains some common methods and processes to help you minimize and eliminate it from your assessments. He references the “Fraud Triangle” created by famed criminologist Donald Cressey to explain why people commit fraud. Eric examines the three elements in the triangle: Motivation, Rationalization and Opportunity. He also offers some practical tips for deterring fraud.

If you are interested in this and many other issues surrounding assessment be sure to check out Eric’s blog.