The Power of Open: Questionmark’s open assessment platform

Posted by Steve Lay

In the beginning there was CVS, then there was SVN and now there’s Git.  What am I talking about?  These are all source code control systems, systems that are used to store computer source code in a way that preserves the complete version history and provides a full audit trail covering the who, what, when and why changes were made.

When we think of open source software we tend to think of the end product: a freely downloadable program that you can run on your computer or even a complete computer operating system in the case of Linux.  But to open source developers, open source is about more than this ‘free beer’ model of sharing software.  Open source software is shared at the source code level allowing people to examine the way it works, suggest changes to fix bugs, enhance it or even to modify it for their own purposes.  Getting the most from sharing source code requires more than just sharing an executable or a zip file of the finished product, open source developers need to open up their source code control systems too.

For years there have been services that provide a cloud-based alternative to  hosting your own source code.  The SourceForge system enjoyed many years of dominance but more recently it’s advertising sponsored model has seen it fall out of favour.

Most new projects are now created on a service called GitHub, which promises  free hosting of open source projects on a service funded by paying customers who are developing projects privately on the same platform.  The success of GitHub has been phenomenal – Google closed down its own rival service (Google Code) largely because of GitHub’s success.  In fact, GitHub is rapidly becoming a ‘unicorn’ with all the associated growing pains.  GitHub makes it easy to collaborate on projects too with its issue tracking system and user friendly tools for proposing changes (known as ‘pull requests’).

With GitHub as the de facto place to publish and share source code, it makes sense for Questionmark to use it to complement our Open Assessment Platform.  We have published source code illustrating how to use our APIs for many years and even publish the complete source to some of our connectors.  Putting new projects on GitHub means providing sample code in the most transparent and developer-friendly way possible.

Questionmark’s GitHub page lists all the projects we own.  For example, when we first brought out our OData APIs we published the sample reportlet code in the OData Reportlet Samples project.  You can experiment with these same examples running live in our website’s developer pages.

Recently we’ve gone a step further in opening up our assessment platform.  We’ve started publishing our API documentation via GitHub too!  Using a new feature of the GitHub platform we’re able to publish the documentation directly from the source control system itself.  That means you always get access to the latest documentation.

Opening up our API documentation in this way makes it easier for developers to engage with our platform.  Why not check out the documentation project.  If you’re already a GitHub user you could ‘watch’ it to get notified when we make changes.  You can even submit issues or send us ‘pull requests’ if you have suggestions for improvement.

With GitHub as the de facto place to publish and share source code, it makes sense for Questionmark to use it to complement our Open Assessment Platform.  We have published source code illustrating how to use our APIs for many years and even publish the complete source to some of our connectors.  Publishing this source code helps our customers and partners by providing working examples of how to integrate with our platform as well as providing complete transparency for our connectors allowing customers to audit the code before they run it on their own systems.  Putting new projects on GitHub means providing sample code in the most transparent and developer-friendly way possible.

Online Proctoring – An Invasion of Privacy?

Steve Lay Headshot

Posted by Steve Lay

Many organisations looking to expand their online offerings now use a new method to securely deliver high-stakes exams online: Online proctoring. A live proctor uses your computer’s webcam to observe you taking the test, to ensure its integrity. To make sure you work alone, the proctor asks you to scan your webcam around the room you are in. The proctors also asks you to show photo ID to verify your identity and will use screen-sharing technology to view your computer screen. In addition, secure browser software can sometimes be used to restrict other computer applications (such as opening a web browser) to restrict a test-taker from accessing digital resources.

Being watched in this way during an online exam often poses questions about privacy…

Is online proctoring an invasion of privacy? Do proctors still have access to your computer after the exam is complete? What sort of things can they access while you’re taking the exam? Can they access your files and identifiable information?

A video link with an online proctor invades no more privacy than taking an exam at a traditional face-to-face test centre. In many cases, allowing a proctor to see everything on your computer screen is just like a proctor at a test centre who can look over your shoulder, see your computer screen and prevent any restricted behavior.  But some online proctoring systems go even further, providing proctors with full control over a candidate’s computer.

Having a proctoring service take control of a candidate’s computer can often be quite helpful.  For instance a proctor who is trained in diagnosing and correcting setup issues can help speed up a process and can quickly resolve problems with the video or audio on the computer. A proctor can also guide the candidate through the exam software, in some cases entering special purpose access credentials that have not previously been made available to the candidate.

Although screen sharing and remote control solutions can be used with Questionmark Online Proctoring, there are alternatives for situations in which such far-reaching access to the candidate’s laptop is inappropriate. Using Questionmark Secure in conjunction with Questionmark OnDemand supports a special mode for online proctoring that gives the proctor limited proxy controls instead of complete control over the machine. For example the proctor can manage the running of the assessment without having control over the participant’s machine. The sense of ‘control’ that many proctoring solutions require here is  similar to popular screen sharing systems that allow you to “Give Control” or “Request Control”. Questionmark Online Proctoring does not require this, because the proctor is connected directly to Questionmark’s service and can manage the exam without going ‘through’ the participant’s computer.

In addition to the privacy advantages of these proxy controls for the candidate, this arrangement also enables the test content to be kept hidden from the proctor. This could provide advantages to the test provider over and above what can be achieved even in a test centre.  The proxy controls allow the proctor to pause the test, add extra time and even terminate the test completely. Meanwhile, Questionmark Secure takes care of monitoring the local computer for signs of misuse and flagging or preventing attempts to cheat.  Questionmark Secure can be audited and installed by a trusted system administrator for a company-owned laptop without having to provide the same permissions to the end user.  Questionmark Secure does not install keylogging software, or any other persistently active service.  It is only active during the exam process itself.

Interested in learning more about Online Proctoring? I will be presenting a session on ensuring exam integrity with online proctoring at Questionmark Conference 2016: Shaping the Future of Assessment in Miami, April 12-15.

There’s only one day left to take advantage our earl-bird savings…click here to register and learn more about this important learning event. See you in Miami!

Unlocking website security

Steve Lay HeadshotPosted by Steve Lay

As a product manager at Questionmark, one of the questions that I’m increasingly being asked is about support for specific versions of SSL and TLS. These abbreviations refer to different flavours of the ‘https’ protocol that keeps your web browsing secure. Questionmark’s OnDemand service no longer supports the older SSL protocol. To understand why, read on…

In this post I’ll focus on the privacy aspect of secure websites only —the extent to which communication is protected from eavesdroppers. Issues of trust are just as important, but I’ll have to discuss those in a future post.

Most browsers display a padlock icon by the web address or the site name to indicate that communication between your browser and the server is encrypted for privacy. Just as with real padlocks, though, there are stronger and weaker forms of encryption. The difference is too subtle for most browsers to show. In practice, browsers adopt a strategy of attempting to use the strongest type of encryption protocol they can, falling back to weaker methods if required. In Internet Explorer you can even configure these settings under the Advanced tab of your internet options:

qm comp 1As you can see, there are five different encryption protocols listed, in increasing order of strength. Generally speaking, TLS is better than SSL and more recent versions of TLS are better still. Published attacks on these protocols typically enable someone who can view network traffic to decrypt some or even all of the information passing over the ‘secure connection’. This type of scenario is called a ‘man in the middle attack’ because the eavesdropper stands in between your browser and the website it is communicating with.

If your browser always chooses the best encryption available, why would you want to configure the specific protocols it supports? Unfortunately, the very first part of the communication between your browser and the website is more vulnerable. The two systems have to agree on an encryption protocol to use before they can be truly private. In some special cases it is possible for a man in the middle to intervene and force a weaker protocol to be negotiated. By configuring your browser to support only stronger protocols, you can ensure that your browser is never tricked this way.

Here at Questionmark, we care about your security too! If a protocol like SSLv3 is considered vulnerable to interception, shouldn’t the server refuse to use it as well? Yes, it should. In fact, we don’t support SSL versions 2 and 3 for this very reason.

For this blog post I’ve focused on the most visible aspect of the security protocol. In practice, there lots of subtle differences in the way each protocol can be configured. If you use Google’s Chrome browser you can click on the padlock to reveal information about connection security.

qm compNotice that this connection uses TLS 1.2, but there is even more detail reported concerning the specific cryptographic algorithms used. Sites like www.ssllabs.com have almost 50 separate check points that they can report on for a public-facing secure website! Staying on top of all this configuration complexity is critical to keeping websites secure.

Unfortunately, sometimes we have to strengthen security in such a way that compatibility with older browsers is sacrificed. For example, according to the latest simulation results, Internet Explorer version 6 (running on Windows XP) is no longer able to successfully negotiate a secure connection with our OnDemand service.

In practice, an overwhelming majority of users use more modern browsers (or have access to one), so the web remains both secure and usable. Perhaps a greater cause of concern is older applications that are integrated with our APIs. It is just as important to keep these applications up to date. For example, applications that use older versions of Java, such as Java 6 or have their Java runtime configuration options set inappropriately might have problems communicating to the same high standards. If you are running a custom integration and are concerned about future compatibility, please get in touch.

This is a developing field. New ways of exploiting older protocols and cryptographic algorithms are being found by security researchers all the time, and the bad guys aren’t far behind. Our security specialists at Questionmark constantly monitor best practice and update the configuration of our OnDemand service to keep your communications safe.

Exams and social media: is it really spying?

Steve Lay HeadshotPosted by Steve Lay

While I was traveling back from our US Users Conference several weeks ago, a debate was raging on social media following news that a testing company had been monitoring Twitter to detect evidence of leaked content. The Guardian newspaper, for example, reported that a New Jersey superintendent had found this ‘disturbing’.

In case you haven’t read about this case, here are the basics: after school, a student tweeted information about a test administered earlier that day. An automated Web monitoring system discovered the tweet, and the school was notified. The student later deleted the offending tweet.

According to the test provider, administrators are supposed to tell participants that sharing any test question online is prohibited. It isn’t clear from the press reports whether this warning was issued prior to the test or whether the student would have considered the tweet prohibited or not. Whatever the case may be, enough information was shared to trigger the automated warning.

Perhaps more interesting than the story itself is the reaction to it. Strong words have been used, but should monitoring social media really be regarded as spying?

The monitoring of online forums to check for exam leaks is not new. It goes back to the very earliest days of the Internet. When I first read about this case my first reaction was that this type of thing is happening all the time. Indeed, brand owners are constantly monitoring social media to help them understand the public’s reaction to their products and services and to help them target their advertising more effectively. Copyright owners also monitor the web to check for infringement. Trademark owners must pro-actively monitor for misuse to prevent their trademarks from becoming unenforceable. So if an organization has such rights, wouldn’t monitoring the web–including social media–to enforce them surely be expected?

This assumption is probably naive. Many people are not aware that this information is available in a form that can be subscribed to. They do not understand the subtle difference between a comment being made in a ‘public place’ like twitter and it being instantly discoverable. In our everyday experience, a conversation that happens in a public place like a café or store is not recorded, transcribed and then made instantly available to business partners of the venue. In this case, the student, the student’s parents and even the superintendent were surprised and shocked by the level of surveillance. They reacted as if a private conversation had been overheard.

It is interesting to contrast this recent case with one reported by Techcrunch in 2009, when information from Facebook was used to hold students to account for cheating. But in the Facebook case, the information was discovered by other students and brought to the attention of the test authorities. Why would the students do that? Likely because test takers are key stakeholders too! If cheating becomes commonplace, then the test will become worthless. So both the test publisher and the test taker have an interest in ensuring fair practice.

Coming back to the rogue tweet, what’s frustrating here is that there is no suggestion that the test taker was trying to cheat or to help someone else cheat. I haven’t seen the 140 characters in question, but it seems likely that the tweet was just a trivial extension of the type of verbal conversation that people frequently have after taking tests.

The mismatch in privacy expectations and the feeling that the student was being accused of malpractice were a toxic mix. Both of these can be avoided.

When monitoring people using CCTV or similar technologies, it is good practice to inform people that they are being monitored, and for what purpose. In many jurisdictions this may also be a legal requirement. Likewise, why not inform test takers of the type of monitoring that is taking place and why? This may have the added advantage of helping to inform them about the risks to their own privacy that over-sharing on social media can pose.

Also, when issues are flagged by monitoring services, test publishers should think carefully about any follow-up actions. Are these actions consistent with the stated purpose of the monitoring? Are they proportionate? Remember, the test taker and the test publisher should be on the same side!

Interact with your data: Looking forward to Napa

Steve Lay HeadshotPosted by Steve Lay

It’s almost time for the Questionmark Users Conference, which this year is being held in Napa, California. As usual there’s plenty on the program for delegates interested in integration matters!

At last year’s conference we talked a lot about OData for Analytics, (which I have also written about here: What is OData, and why is it important? ). OData is a data standard originally created by Microsoft but now firmly embedded in the open standards community through a technical group at OASIS. OASIS have taken on further development, resulting in the publication of the most recent version, OData 4.

This year we’ve built on our earlier work with the Results OData API to extend our adoption of OData to our delivery database, but there’s a difference. Whereas the Results OData API provides access to data, the data exposed from our delivery system supports read and write actions, allowing third-party integrations to interact with your data during the delivery process.

Why would you want to do that?

Some assessment delivery processes involve actions that take place outside the Questionmark system. The most obvious example is essay grading. Although the rubrics (the rules for scoring) are encoded in the Questionmark database, it takes a human being outside the system to follow those rules and to assign marks to the participant. We already have a simple scoring tool built directly in to Enterprise Manager but for more complex scoring scenarios you’ll want to integrate with external marking tools.

The new Delivery OData API provides access to the data you need, allowing you to read a participant’s answers and write back the scores using a simple Unscored -> Saved -> Scored workflow. When the result is placed in the final status, the participant’s result is updated and will appear with the updated scores in future reports.

I’ll be teaming up with Austin Fossey, our product owner for reporting, and Howard Eisenberg, our head of Solution Services, to talk at the conference about Extending Your Platform, during which we’ll be covering these topics. I’m also delighted that colleagues from Rio Salado College will also be talking about their own scoring tool that is built right on top of the Delivery OData API.

I look forward to meeting you in Napa but if you can’t make it this year, don’t worry, some of the sessions will be live-streamed. Click here to register so that we can send you your login info and directions. And you can always follow along with social media by following and tweeting with @Questionmark.

Acronyms, Abbreviations and APIs

Steve Lay HeadshotPosted by Steve Lay

As Questionmark’s integrations product owner, it is all too easy to speak in acronyms and abbreviations. Of course, with the advent of modern day ‘text-speak,’ acronyms are part of everyday speech. But that doesn’t mean everyone knows what they mean. David Cameron, the British prime minister, was caught out by the everyday ‘LOL’ when it was revealed during a recent public inquiry that he’d used it thinking it meant ‘lots of love’.

In the technical arena things are not so simple. Even spelling out an acronym like SOAP (which stands for Simple Object Access Protocol) doesn’t necessarily make the meaning any clearer. In this post, I’m going to do my best to explain the meanings of some of the key acronyms and abbreviations you are likely to hear talked about in relation to Questionmark’s Open Assessment Platform.

API

At a recent presentation (on Extending the Platform), while I was talking about ways of integrating with Questionmark technologies, I asked the audience how many people knew what ‘API’ stood for. The response prompted me to write this blog article!

The term, API, is used so often that it is easy to forget that it is not widely known outside of the computing world.

API stands for Application Programming Interface. In this case the ‘application’ refers to some external software that provides functionality beyond that which is available in the core platform. For example, it could be a custom registration application that collects information in a special way that makes it possible to automatically create a user and schedule them to a specified assessment.

The API is the information that the programmer needs to write this registration application. ‘Interface’ refers to the join between the external software and the platform it is extending. (Our own APIs are documented on the Questionmark website and can be reached directly from developer.questionmark.com.)

APIs and Standards

APIs often refer to technical standards. Using standards helps the designer of an API focus on the things that are unique to the platform concerned without having to go into too much incidental detail. Using a common standard also helps programmers develop applications more quickly. Pre-written code that implements the underlying standard will often be available for programmers to use.

To use a physical analogy, some companies will ask you to send them a self-addressed stamped envelope when requesting information from them. The company doesn’t need to explain what an envelope is, what a stamp is and what they mean by an address! These terms act a bit like technical standards for the physical world. The company can simply ask for one because they know you understand this request. They can focus their attention on describing their services, the types of requests they can respond to and the information they will send you in return.

QMWISe

QMWISe stands for Questionmark Web Integration Services Environment. This API allows programmers to exchange information with Questionmark OnDemand software-as-a-service or Questionmark Perception on-premise software. QMWISe is based on an existing standard called SOAP. (see above)

SOAP defines a common structure used for sending and receiving messages; it even defines the concept of a virtual ‘envelope’. Referring to the SOAP standard allows us to focus on the contents of the messages being exchanged such as creating participants, creating schedules, fetching results and so on.

REST

REST stands for REpresentational State Transfer and must qualify as one of the more obscure acronyms! In practice, REST represents something of a back-to-basics approach to APIs when contrasted with those based on SOAP. It is not, in itself, a standard but merely a set of stylistic guidelines for API designers defined by an academic paper written by Roy Fielding, a co-author of the HTTP standard (see below).

As a result, APIs are sometimes described as ‘RESTful’, meaning they adhere to the basic principles defined by REST. These days, publicly exposed APIs are more likely to be RESTful than SOAP-based. Central to the idea of a RESTful API is that the things your API deals with are identified by a URL (Uniform Resource Locator), the web’s equivalent of an address. In our case, that would mean that each participant, schedule, result, etc. would be identified by its own URL.

HTTP

RESTful APIs draw heavily on HTTP. HTTP stands for HyperText Transfer Protocol. It was invented by Tim Berners-Lee and forms one of the key inventions that underpin the web as we know it. Although conceived as a way of publishing HyperText documents (i.e., web pages), the underlying protocol is really just a way of sending messages. It defines the virtual envelope into which these messages are placed. HTTP is familiar as the prefix to most URLs.

OData

Finally this brings me to OData. OData just stands for Open Data. This standard makes it much easier to publish RESTful APIs. I recently OData in the post, What is Odata, and why is it important?

Although arguably simpler than SOAP, OData provides an even more powerful platform for defining APIs. For some applications, OData itself is enough, and tools can be integrated with no additional programming at all. The PowerPivot plugin for Microsoft Excel is a good example. Using Excel you can extract and analyse data using the Questionmark Results API (itself built on OData) without any Questionmark-specific programming at all.

For more about OData, check out this presentation on Slideshare.