Eight ways to check if security is more than skin deep

Picture of computer and padlockJohn Kleeman HeadshotPosted by John Kleeman

The assessment industry has always been extremely careful about exam security and ways to prevent cheating. As cloud and online assessment takes over as delivery models, it’s critical we all deeply embed IT security in our culture to ensure that computer vulnerabilities don’t leak sensitive data or disrupt the integrity of the assessment process.

Many years ago, Questionmark realized that data protection and IT security were critical to our success. We re-formed our culture to make security a priority. We followed our own path and looked for opportunities to learn from others such as Bill Gates and his famous trustworthy computing memo, part of which is quoted below:

… when we face a choice between adding features and resolving security issues, we need to choose security. Our products should emphasize security right out of the box, and we must constantly refine and improve that security as threats evolve. …  These principles should apply at every stage of the development cycle of every kind of software we create …

Questionmark understands that we’re in an arms race. We stay vigilant and look for opportunities to improve our security. Here are eight key ways in which we have embedded security deep within our company. If you are an assessment provider, we’d encourage you to find your own way to follow suit. And if you are a customer, here are eight questions you can ask to identify whether an assessment provider is truly working to be as secure as it can,  instead of just claiming to be secure when in fact security is only skin deep.

1. Who does the security function report to?

At Questionmark our security officer reports directly to me as Questionmark Chairman. If security reports directly into IT or product development, a security concern might be overruled by operational need. We’ve found this separation very helpful to ensure security gets listened to throughout the organization.

2. Would a security flaw hold up a release?

In any sensible company, this has to be true. Feature improvements in software are important, but if there is a serious security issue, it needs to be fixed first. Developers need to know that they can’t make a release unless it is secure.

3. How do you check your employees know about security?

Questionmark trains all our employees on data security but how do we know they understand? We practice what we preach and everyone from senior management to sales to accounting to developers needs to take and pass a data security test every year to check understanding. I’d encourage everyone in the assessment industry to follow this approach.

4. How deep is your team’s knowledge of IT security?

SaaS security is complex. There are many layers to security and any weakness can lead to a vulnerability. Equally throwing resources in the wrong place won’t really help. We are fortunate to have at least half a dozen experts within Questionmark who have deep knowledge of and passion for different aspects of security. This helps us get things right,.

5. Is your ecosystem secure?

Every company operates in an ecosystem , and it’s the ecosystem that needs to be secure. Questionmark works with our suppliers, subcontractors and partners to help them to be secure, including offering training and advice. We even want our competitors to be secure as any breaches in the assessment industry would be hurtful to all.

6. How transparent and open are you on your security?

Security by obscurity is not secure. Questionmark shares information on the security of our OnDemand service in white papers (Security of Questionmark’s US OnDemand Service and Security of Questionmark’s EU OnDemand Service) and have “red papers” which describe our security and business continuity planning in detail, available under NDA to prospective customers. The review process as customers ask questions about these provides comfort for customers and input to us to improve our security.

7. What kinds of external review do you allow?

As we shared in Third-party audits verify our platform’s security, we run regular penetration tests by a third party company, Veracode on Questionmark OnDemand. We are also fortunate to have many customers who care deeply about security and undertake their own audits and reviews by experts. We welcome such review and learn from it to improve our own security.

8. Are you completely satisfied with your security?

imageAbsolutely not. There is an arms race happening in the security world. Hackers and other bad actors are increasing their capabilities and however good you are, if you rest on your laurels, the arms race will overtake you. See for example the graph to the right from Verizon showing the increase in breaches over time.

Questionmark, like other good SaaS companies, has a policy of continual improvement – we want to be much better each year than the last.

This video provides an overview of how Questionmark builds security into its products from day one. Watch below:

Do certification exams give business benefit?

Posted by John Kleeman

Increasing numbers of technology vendors run certification programmes to help customers, employees and partners demonstrate competence in using or advising on the vendor’s technology. This is common in IT, in medical equipment, in the automotive industry and in many other high-tech industries.

Certification is an area where all stakeholders seem to be “winners”:

  • Vendors who set up certification programmes gain by being able to define the skill sets and knowledge that people deploying their technology need and encouraging stakeholders to develop the knowledge and skills and so deploy the technology more successfully for customers.
  • Participants benefit from certification as a way to learn and develop skills and demonstrate their competence, and it often helps in their career path.
  • Customers and users of the technology benefit from more effective deployment by being able to ensure the skills of experts deploying the technology and being more likely to get a successful implementation.
  • Employers of test-takers gain from their employees being more capable.

But how real is the benefit? How can you know if a well-designed and well-implemented certification programme will lead to improved performance?

There is some powerful evidence about this from an IDC study a few years back as reported on by Network World. This study looked at the benefit of certifications within IT network administration – surveying more than 1,000 IT managers. You can see some of the results in the chart below. For instance on average, unscheduled network downtime was about 20% lower at organizations with more certified IT staff.

Impacct of certifications on network administration

This study related to one particular field of IT, but it seems likely that in any technical field, providing you follow good practice in developing your certification programme, similar results should apply. Therefore certification is likely to provide material business benefits.

Questionmark is certainly seeing a lot of interest from customers looking to deliver certifications online, and I’ll talk about that in a follow-up post.

Assessment Standards Part Three: ISO 23988

john_smallPosted by John Kleeman

This is the third of a series of blog posts on standards that impact assessment. I’ve participated in many standards projects over the years, but there’s only one standard which I can be pretty sure would never have happened without my involvement.

Around the turn of the millennium I had a significant birthday, and rather than do the usual work tasks, I decided to use the day for something more creative. It was just around the formation of a brand new International Standards (ISO) working group on learning technology (SC36) and I was part of a newly formed British Standards Committee that shadowed the ISO committee. We were looking for new standards to develop and it was about the time that using computers and the Internet for delivering assessments was really coming of age. Lots of people were using Questionmark software or other software to deliver assessments and as people learned, they made mistakes which could cause unfairness and pain.

I thought it would be great to have a Code of Practice on how to use computers to deliver assessments.  If this could be a standard, it would encourage everyone to follow good practice and would make things fairer and better for everyone using assessments.  I would also allow everyone to benefit from the experience of the best practitioners.

So I proposed the idea to the UK committee and after a while I led a panel of many experts in assessment to come up with what was then called BS 7988 – Code of Practice for the use of Information Technology (IT) in the Delivery of Assessments. Many wiser people than I contributed to the standard: assessment experts, technology experts and educational experts.  BS 7988 was published in 2002, and in due course it was taken by the BSI to ISO to become (after some editing) an international standard ISO 23988.

The standard contains guidance and context for using IT to deliver assessments. Due to the vagaries of international standards economics, you have to pay to buy the standard so I’m limited in how I can quote from it.  However, I hope that ISO won’t mind me quoting one illustrative clause, which applies to assessments that are invigilated or proctored:

    At least one invigilator should be present in the room throughout the assessment
    session. If there is a single invigilator, he/she should be able to summon help (including
    technical help) quickly if needed. Unless there is only one candidate, the invigilator should
    not be distracted from invigilation duties by having to provide technical help.

Not rocket science, but useful common sense. And there are 45 pages of useful material in the standard with lots of sensible guidelines.

As the saying goes, “What’s the difference between theory and practice?  In theory there is none, but in practice there is!” ISO 23988 encapsulates a lot of good practice in delivering assessments and puts it in a standard code of practice for everyone to pick from or follow.