5 Things I Learned at the European Association of Test Publishers Conference Last Week

Posted by John Kleeman

I just attended the Association of Test Publisher’s European conference (EATP), held last week in Madrid, and wanted to share some of what I learned.

The Association of Test Publishers (ATP) is the trade association for the assessment industry and promotes good practice in assessment. Questionmark have been members for a long time and I am currently on their board of directors. The theme of the conference was “Transforming Assessments: Challenge. Collaborate. Inspire.”

Panel at European Association of Test Publishers

As well as seeing a bit of Madrid (I particularly enjoyed the beautiful Retiro Park), here are some things I learned at the conference. (These are all my personal opinions, not endorsed by Questionmark or the ATP).

1. Skills change. One area of discussion was skills change. Assessments are often used to measure skills, so as skills change, assessments change too. There were at least three strands of opinion. One is that workplace skills are changing rapidly – half of what you learn today will be out of date in five years, less if you work in technology. Another is that many important skills do not change at all – we need to collaborate with others, analyze information and show emotional resilience; these and other important skills were needed 50 years ago and will still be needed in 50 years’ time. And a third suggested by keynote speaker Lewis Garrad is that change is not new. Ever since the industrial revolution, there has been rapid change, and it’s still the case now. All of these are probably a little true!

2. Artificial Intelligence (AI). Many sessions at the conference covered AI. Of course, a lot of what gets called AI is in fact just clever marketing of smart computer algorithms. But nevertheless, machine learning and other things which might genuinely be AI are definitely on the rise and will be a useful tool to make assessments better. The industry needs to be open and transparent in the use of AI. And in particular, any use of AI to score people or identify anomalies that could indicate test cheating needs to be very well built to defend against the potential of bias.

3. Debate is a good way to learn. There were several debates at the conference, where experts debated issues such as performance testing, how to detect fraud and test privacy vs security, with the audience voting before and after. As the Ancient Greeks knew, this is a good format for learning, as you get to see the arguments on both sides presented with passion. I’d encourage others to use debates for learning.

4. Privacy and test security genuinely need balance. I participated in the privacy vs test security debate, and it’s clear that there is a genuine challenge balancing the privacy rights of individual test-takers and the needs of testing organizations to ensure results are valid and have integrity. There is no single right answer. Test-taker rights are not unlimited. And testing organizations cannot do absolutely anything they want to ensure security. The growing rise of privacy laws including the GDPR has brought discussion about this to the forefront as everyone seeks to give test-takers their mandated privacy rights whilst still being able to process data as needed to ensure test results have integrity. A way forward seems to be emerging where test-takers have privacy and yet testing organizations can assert legitimate interests to resist cheating.

5. Tests have to be useful as well as valid, reliable and fair. One of the highlights of the conference was a CEO panel, where Marten Roorda, CEO of ACT, Norihisa Wada, a senior executive at EduLab in Japan, Sangeet Chowfla, CEO of the Graduate Management Admission Council and Saul Nassé, CEO of Cambridge Assessment gave their views on how assessment was changing. I moderated this panel (see picture below) and it was great to hear these very smart thought leaders talk of the future.  There is widespread agreement that validity, reliability and fairness are key tenets for assessments , but also a reminder that we also need “efficacy” – i.e. that tests need to be useful for their purpose and valuable to those who use them.

There was a huge amount of other conference conversations including sessions on online proctoring, test translation, the update to the ISO 10667 standard, producing new guidelines on technology based assessment and much, much more.

I found it challenging, collaborative and inspiring and I hope this blog gives you a small flavor of the conference.

Six predictions now the GDPR is in place

Posted by John Kleeman
So the European GDPR is in place now. Questionmark like most other companies has been working hard in the last two years to ensure we are compliant and that our customers in and outside Europe can be compliant with the GDPR. See our trust center or summary for information on Questionmark’s compliance.

Is it all done and dusted? My email inbox seems to have a few less promotional emails in it. But is this because of the holiday weekend or have companies really taken my name off their mailing lists? Here are six predictions for what we’ll see going forwards with the GDPR.

1. The May 25th 2018 date will matter much less going forwards than backwards

A picture of a dog with a Christmas hatCompanies have been rushing to meet the May 25th date, but GDPR and privacy is a destination not a journey. There is a famous slogan “a dog is for life not just for Christmas” encouraging people to look after their dog and not just buy it as a cute puppy. Similarly the GDPR is not just something you get compliant with and then ignore. You need to include privacy and compliance in your processes forever.

No one will care much whether you were compliant on May 25th 2018. But everyone will care whether you are meeting their privacy needs and following the law when they interact with you.

2. History will judge the GDPR as a watershed moment where privacy became more real

Nevertheless I do think that history will judge the GDPR as being a seminal moment for privacy. Back in the early 2000s, Microsoft popularized the concept of security by design and security by default when they delayed all their products for a year as they improved their security. Nowadays almost everyone builds security into their systems and makes it the default because you have to to survive.

Similarly the GDPR encourages us to think of privacy when we design products and to make privacy the default not an afterthought. For example, when we collect data, we should plan how long to keep it and how to erase it later. I suspect in ten years time, privacy by design will be as commonplace as security by design – and the GDPR will be the key reason it became popular.

3. Many other jurisdictions will adopt GDPR like laws

Although the GDPR is over-complex, it has some great concepts in it, that I’m sure other countries will adopt. It is appropriate that organizations have to take care about processing peoples’ data. It is appropriate that when you pass people’s data onto a third party, there should be safeguards. And if you breach that data, it is appropriate that you should have to be held accountable.

We can expect lawmakers in other countries to make GDPR-like laws.

4. Supply chain management will become more important

Diagram showing one data controller with two data processors. One data processor has two sub-processors and one data processor has one sub-processorUnder the GDPR, a Data Controller contracts with Data Processors and those Data Processors must disclose their Sub-processors (sub-contractors). There is positive encouragement to choose expert Data Processors and Sub-processors and there are consequences if processors fail their customers. This will encourage organizations to choose reputable suppliers and to review processors down the chain to make sure that everyone is following the rules. Choosing suppliers and Sub-processors that get themselves audited for security, e.g. under ISO 27001, is going to become more commonplace.

This will mean that some suppliers who do not have good enough processes in place for security, privacy and reliability will struggle to survive.

5. People will be the biggest cause of compliance failures

Organizations set up processes and procedures and put in place systems and technology to run their operations, but people are needed to design and run those processes and technology. Some GDPR compliance failures are going to be down to technology failures, but I predict the majority will be down to people. People will make mistakes or judgement errors and cause privacy and GDPR breaches.

If you are interested in this subject, Amanda Maguire of SAP and I gave a webinar last week entitled “GDPR is almost here – are your people ready?” which should shortly be available to view on the SAP website. The message we shared is that if you want to stay compliant with the GDPR, you need to check your people know what to do with personal data. Testing them regularly is a good way of checking their knowledge and understanding.

6. The GDPR and privacy concerns will encourage more accurate assessments

Last but not least, I think that the GDPR will encourage people to expect more accurate and trustworthy tests and exams. The GDPR requires that we pay attention to the accuracy of personal data; “every reasonable step must be taken to ensure that personal data that are inaccurate … are erased or rectified without delay”.

There is a strong argument this means that if someone creates a test or exam to measure competence, that the assessment should be accurate in what it claims to measure. So it needs to be authored using appropriate procedures to make it valid, reliable and trustworthy. If someone takes an assessment which is invalid or unfair, and fails it, they might reasonably argue that the results are not an accurate indication of their competence and so that personal data is inaccurate and needs correcting.

For some help on how you can make more accurate assessments, check out Questionmark white papers at www.questionmark.com/learningresources including “Assessment Results You Can Trust”.

 

 

Online Proctoring – An Invasion of Privacy?

Steve Lay Headshot

Posted by Steve Lay

Many organisations looking to expand their online offerings now use a new method to securely deliver high-stakes exams online: Online proctoring. A live proctor uses your computer’s webcam to observe you taking the test, to ensure its integrity. To make sure you work alone, the proctor asks you to scan your webcam around the room you are in. The proctors also asks you to show photo ID to verify your identity and will use screen-sharing technology to view your computer screen. In addition, secure browser software can sometimes be used to restrict other computer applications (such as opening a web browser) to restrict a test-taker from accessing digital resources.

Being watched in this way during an online exam often poses questions about privacy…

Is online proctoring an invasion of privacy? Do proctors still have access to your computer after the exam is complete? What sort of things can they access while you’re taking the exam? Can they access your files and identifiable information?

A video link with an online proctor invades no more privacy than taking an exam at a traditional face-to-face test centre. In many cases, allowing a proctor to see everything on your computer screen is just like a proctor at a test centre who can look over your shoulder, see your computer screen and prevent any restricted behavior.  But some online proctoring systems go even further, providing proctors with full control over a candidate’s computer.

Having a proctoring service take control of a candidate’s computer can often be quite helpful.  For instance a proctor who is trained in diagnosing and correcting setup issues can help speed up a process and can quickly resolve problems with the video or audio on the computer. A proctor can also guide the candidate through the exam software, in some cases entering special purpose access credentials that have not previously been made available to the candidate.

Although screen sharing and remote control solutions can be used with Questionmark Online Proctoring, there are alternatives for situations in which such far-reaching access to the candidate’s laptop is inappropriate. Using Questionmark Secure in conjunction with Questionmark OnDemand supports a special mode for online proctoring that gives the proctor limited proxy controls instead of complete control over the machine. For example the proctor can manage the running of the assessment without having control over the participant’s machine. The sense of ‘control’ that many proctoring solutions require here is  similar to popular screen sharing systems that allow you to “Give Control” or “Request Control”. Questionmark Online Proctoring does not require this, because the proctor is connected directly to Questionmark’s service and can manage the exam without going ‘through’ the participant’s computer.

In addition to the privacy advantages of these proxy controls for the candidate, this arrangement also enables the test content to be kept hidden from the proctor. This could provide advantages to the test provider over and above what can be achieved even in a test centre.  The proxy controls allow the proctor to pause the test, add extra time and even terminate the test completely. Meanwhile, Questionmark Secure takes care of monitoring the local computer for signs of misuse and flagging or preventing attempts to cheat.  Questionmark Secure can be audited and installed by a trusted system administrator for a company-owned laptop without having to provide the same permissions to the end user.  Questionmark Secure does not install keylogging software, or any other persistently active service.  It is only active during the exam process itself.

Interested in learning more about Online Proctoring? I will be presenting a session on ensuring exam integrity with online proctoring at Questionmark Conference 2016: Shaping the Future of Assessment in Miami, April 12-15.

There’s only one day left to take advantage our earl-bird savings…click here to register and learn more about this important learning event. See you in Miami!