The Power of Open: Questionmark’s open assessment platform

Posted by Steve Lay

In the beginning there was CVS, then there was SVN and now there’s Git.  What am I talking about?  These are all source code control systems, systems that are used to store computer source code in a way that preserves the complete version history and provides a full audit trail covering the who, what, when and why changes were made.

When we think of open source software we tend to think of the end product: a freely downloadable program that you can run on your computer or even a complete computer operating system in the case of Linux.  But to open source developers, open source is about more than this ‘free beer’ model of sharing software.  Open source software is shared at the source code level allowing people to examine the way it works, suggest changes to fix bugs, enhance it or even to modify it for their own purposes.  Getting the most from sharing source code requires more than just sharing an executable or a zip file of the finished product, open source developers need to open up their source code control systems too.

For years there have been services that provide a cloud-based alternative to  hosting your own source code.  The SourceForge system enjoyed many years of dominance but more recently it’s advertising sponsored model has seen it fall out of favour.

Most new projects are now created on a service called GitHub, which promises  free hosting of open source projects on a service funded by paying customers who are developing projects privately on the same platform.  The success of GitHub has been phenomenal – Google closed down its own rival service (Google Code) largely because of GitHub’s success.  In fact, GitHub is rapidly becoming a ‘unicorn’ with all the associated growing pains.  GitHub makes it easy to collaborate on projects too with its issue tracking system and user friendly tools for proposing changes (known as ‘pull requests’).

With GitHub as the de facto place to publish and share source code, it makes sense for Questionmark to use it to complement our Open Assessment Platform.  We have published source code illustrating how to use our APIs for many years and even publish the complete source to some of our connectors.  Putting new projects on GitHub means providing sample code in the most transparent and developer-friendly way possible.

Questionmark’s GitHub page lists all the projects we own.  For example, when we first brought out our OData APIs we published the sample reportlet code in the OData Reportlet Samples project.  You can experiment with these same examples running live in our website’s developer pages.

Recently we’ve gone a step further in opening up our assessment platform.  We’ve started publishing our API documentation via GitHub too!  Using a new feature of the GitHub platform we’re able to publish the documentation directly from the source control system itself.  That means you always get access to the latest documentation.

Opening up our API documentation in this way makes it easier for developers to engage with our platform.  Why not check out the documentation project.  If you’re already a GitHub user you could ‘watch’ it to get notified when we make changes.  You can even submit issues or send us ‘pull requests’ if you have suggestions for improvement.

With GitHub as the de facto place to publish and share source code, it makes sense for Questionmark to use it to complement our Open Assessment Platform.  We have published source code illustrating how to use our APIs for many years and even publish the complete source to some of our connectors.  Publishing this source code helps our customers and partners by providing working examples of how to integrate with our platform as well as providing complete transparency for our connectors allowing customers to audit the code before they run it on their own systems.  Putting new projects on GitHub means providing sample code in the most transparent and developer-friendly way possible.

How online assessments (quizzes, tests and exams) can help information security awareness and compliance

Posted by John Kleeman

With the rise of data security leakages, most professional organizations are seeking to significantly upscale their cybersecurity to better protect their organization from information security risks. I see an increasing use of online assessments helping information security and thought I’d provide some pointers about this.

There are three main ways in which online quizzes, tests, exams and surveys can aid information security:

  • Testing personnel to check understanding of security awareness and security policies
  • Ensuring and documenting that personnel in security roles are competent
  • Helping measure success against security objectivesNIST logo

Testing on security awareness and knowledge of policies

A cornerstone of good practice in security is training in security awareness. For example, the widely respected NIST 800-53 publication recommends that organizations provide general-purpose and role-based training to personnel as part of initial training and periodically thereafter. If you follow NIST standards, NIST control AT-4 also requires that all security training be documented and records retained.

There is widespread evidence that delivering an assessment is the best way of documenting that training took place, because it doesn’t just document attendance but also understanding of the training. For more explanation, see the Questionmark blog post Proving compliance – not just attendance. The only point of security awareness training is to have the training be understood, so testing to confirm understanding is widespread and sensible.

At Questionmark, we practice what we preach! All our employees have to take a test on data security when they join to check they understand our policies; all employees must also take and pass an updated test each year to ensure they continue to understand.

Ensure that people in security roles are competent

iso 27001The international security standard ISO 27001:2013 requires that an organization determine the necessary competence of personnel affecting information security performance. The organization must also ensures that personnel have such competence and retain evidence of this.

In a large organization with many different security roles, developing and using competence tests for each information security-related role is a good way of measuring and showing competence.  Knowing who is competent in which aspect of security and data protection matters: it ensures that  you are covering appropriate risks with appropriate people. Online testing is an effective way of measuring competence and makes it easy to update competence records by giving periodic tests every six months or annually.

Helping measure information security objectives

PCI logoISO 27001 also requires setting up metrics to measure information security objectives. Results from assessments can be a good metric to use.  Other standards say similar things. For example, the PCI standard widely used for credit card security says in its best practice guide:

“Metrics can be an effective tool to measure the success of a security awareness program, and can also provide valuable information to keep the security awareness program up-to-date and effective”

The PCI guide recognizes that good metrics include “feedback from personnel; quizzes and training assessments”. In my experience, as well as using quizzes and tests to measure knowledge, it also makes sense to use online surveys to assess actual practice by employees and to allow reporting of security concerns.

Testing on information security and data protection is an increasing use case for Questionmark’s trustable SaaS assessment management system, Questionmark OnDemand.  Whichever security standard you are following (ISO 27001, NIST, PCI or one of several others), creating online assessments tailored to measure knowledge of your organization’s policies and procedures using an assessment management system like Questionmark’s can make a useful difference.

Want to enhance exam integrity? Need to ensure compliance? Then don’t miss these webinars!

Posted by Chloe Mendonca

Proxy testing … Cheating …. Content theft. Does the battle ever end? The techniques and technologies being used to compromise the integrity of your test programs are complex and continually evolving. Many organisations are increasingly turning to online proctoring solutions.

Research shows that when properly applied, online proctoring can be as or more effective than traditional test centre proctoring. Besides reducing security risks, it provides test takers the convenience of taking tests from their homes, workplaces or colleges. It also reduces costs not only for administrators but also for test takers. Consider for a moment the testing fees, travel, parking, and the simple time away from work or studies.  In addition to online proctoring’s effectiveness one of the most exciting things is that this approach has the potential to grow as technology becomes more advanced and widespread.

Before implementing online proctoring, organisations should approach with caution, not all online proctoring services are equivalent. While some are designed with high-stakes tests in mind others can actually increase the risk of cheating. So what do you need to consider? What should you be wary of? Is online proctoring suited to your organisation or institution? Join Questionmark for a 45-minute webinar answering these questions and more.

Register for: Enhancing Exam Integrity with Online Proctoring

Online Proctoring is only a small part of achieving exam integrity. For those less familiar with Questionmark’s end-to-end Assessment Management System, find out how security is embedded at every stage of assessment development in our 60-minute demo. Discover the features and functions available to help you securely create, deliver and report on your surveys, quizzes, tests and exams in our introductory webinar. Note, we are also holding this webinar in Dutch.

Register for: An Introduction to Questionmark’s Assessment Management System

Or for organisations working in sectors that demand compliance, learn about 7 ways that assessments can enhance your eLearning and compliance strategy and best practices for using them in our 30-minute German webinar.

Register for: Warum und wie Sie Compliance-Prüfungen nutzen sollten

 

Reminiscing about Santa Fe: Presentations, pictures & the weird and wonderful art house

Posted by Chloe Mendonca

After eagerly looking forward to Questionmark’s most important annual learning event for months, it was over before we even knew it! The Questionmark Conference gave all of us three special days to meet so many of our globally dispersed customers and employees face to face, learn best practices, have fun with one another and discuss new ways to leverage Questionmark’s technologies.

This year I was fortunate enough to be there, and a big highlight was getting a deeper understanding of how others are using Questionmark’s technologies. From our evening networking events to our stimulating panel discussion — which brought together experts from the US State Department, Caterpillar Inc., Scantron and Compass Consultants to discuss best practices for making data work within learning and assessment programs — to more specific breakout sessions, our guest speakers did a wonderful job of sharing lessons learned and best-practice tips.

Todd Horner from Accenture, for example, hosted a great discussion, “Taking the Migraine out of Migration: Accenture’s journey to next-gen authoring.” He spoke about the shared “fear of the unknown” and how to get around change-management challenges. Lauri Buckley and Lindsey Clayton from Caterpillar Inc, delivered an impressive presentation, “A Process to Mastery: Assessments as career development tools,” during which they shared valuable tips about how to effectively design and develop various types of competence assessments, from proficiency tests to validation and observational assessments. You can get the handouts from these presentations and more right here.

For those who couldn’t be there in person, we webcast selected conference sessions — hitting record numbers online. If you joined us for the webcast, got a sense of the Questionmark Conference atmosphere and want to join us in person next year, keep your eyes peeled for our dates and location announcement coming to the blog in the next few months. See the recordings of our selected webcast sessions at: www.questionmark.com/go/2017uconwebcast (Please note you must be logged into the website with your Questionmark username and password).

I’d like to take this opportunity to say a big thank you to all of our wonderful speakers for taking the time to share their knowledge. Without them there would be no conference!

Now for the bit you’ve all been waiting for… conference pictures! To all those who went back to the office struggling to describe the weird and wonderful art house that is Meow Wolf’s House of Eternal Return, hopefully these snaps will make things a little easier 😊  View conference and evening event pictures here on our flickr page.

What did you enjoy most about Questionmark Conference 2017? Leave me a comment below and stay in touch!


Just in case you missed it…

John Kleeman, Questionmark’s Founder & Executive Director reported back 6 good practice tips heard in Santa Fe.

How to manage compliance in a highly regulated world? [30-minute webinar]

Posted by Chloe Mendonca

If your industry demands compliance, then your people need compliance training and certifications. Learning and training records are almost always reviewed during regulatory audits to check that employees have received the required training and their competencies or certifications are up to date and valid. The regular assessment of employee knowledge and competencies ensures you’re always ready for an audit or if something goes wrong.

Did you know…


Source: Brandon Hall Group

Perhaps you already understand the value assessments can bring, but need to convince your management team. Or perhaps you’d like a better view on how to use assessments most effectively to ensure compliance. You’re not alone.

Join us for a 30-minute webinar on Thursday March 9, 2017 to:

  • discuss the critical role assessments play in compliance learning
  • explore the benefits of using assessments before, during and after training
  • find out 7 ways assessments fortify compliance
  • get best practice tips for ensuring valid and reliable assessments

We also have several other webinars you may be interested in:

  • How to write high-quality test items – March 4, 2017

If you’d like best practice tips to improve your test items and ensure they produce fair, valid and reliable results then sign up for this 30-minute webinar.

  • Introduction to Questionmark’s Assessment Management System – Various dates and times

Get an overview of Questionmark’s features and functions in this live demo. We will look at the basics of authoring, delivering and reporting on surveys, quizzes, tests and exams.

FBI and Homeland Security advice on trumping cybersecurity attacks

Posted by John Kleeman

There’s a lot in the news recently about possible cybersecurity attacks on the political process. Here are some thoughts on how we can learn from this and apply it to assessment security.

One of the most interesting documents I’ve read on this subject is the Department of Homeland Security and FBI’s joint analysis report  JAR-16-20296 titled GRIZZLY STEPPE – Russian Malicious Cyber Activity.  This presents evidence on how a cybersecurity attack was made on a US political party in 2016 and gives some practical advice on how others can set up their systems to avoid such attacks.

Whoever the attack was performed by (and there has been some debate about this), the practical advice is useful to anyone who wants to improve their security. I was particularly struck by a section in the report which offered questions to ask your organization to see if they have good cybersecurity practices. I’ve taken the liberty of including the questions in the graphic below:

See Grizzly Steppe report for text here

I’ve shared various sets of security questions in this blog, including Eight ways to check if security is more than skin deep and 24 midsummer questions to ask your assessment software provider, but here are some questions from a very credible source!

I’d encourage you to pose these questions within your organization and with your suppliers to check that you are well protected in case of a cyberattack. Questionmark, like all sensible organizations, believes in continuous improvement in our security, and listening to sources like this analysis informs our improvement.

I hope highlighting the report and these questions helps strengthen your defenses against cybersecurity and acts as a guide in choosing your vendors.

« Previous PageNext Page »