Using the Demographic Report

The Demographic Report was recently added to Questionmark Analytics. Here are the basics:

What it does: The demographic report breaks down results by demographic variables such as language,  course name, location, department, instructor, and job role or military rank. This report can recognize up to 10 demographic variables recorded by an  assessment. Users can review the performance of assessment data broken out by a demographic variable, using data stored in special fields within the database.

Who should use it: Assessment, learning and education professionals can use this report to zero in on test results according to specific demographic information.

How it looks: Graph 1 in this example shows the assessment mean score for the selected demographic. It displays the average percentage score achieved as well as the high and low results.  Graph 2 shows the number of results for the selected demographic and includes an overall calculation of the number of results found.

You can assign several filters to limit the information included in the report:

  • Assessment (mandatory)
  • Group
  • Date
  • Attempt
  • Special field

Using QR codes to direct smartphone users to assessments

QR or “Quick Response” codes offer an easy way to direct users of smartphones, tablets and other mobile devices to your assessments — and thereby to boost participation in surveys, course evaluations, knowledge checks, quizzes and other assessments.Android phone

There are various ways to create QR codes for Questionmark assessments, but here are the basics: 

  • Create your assessment and generate a URL for it
  • Provide your assessment URL to a service such as, orangeqr or to generate a short URL for the assessment
  • Copy the short URL in to your browser address bar and append .qr to the end of it
  • Navigate to that page, where you will see a QR code for you to save for future use

See these PowerPoints on SlideShare for more details about using QR codes to launch Questionmark assessments.

Using the Assessment Completion Time Report

The Assessment Completion Time Report is one of three new reports in Questionmark Analytics. Here are the basics:

What it does: The assessment completion time report graphically monitors how long, on average, it took each participant to complete an assessment in relation to  his or her overall score.  For example, someone who took very  little time to complete the assessment  but achieved a very high score would be flagged. Might this person have had prior knowledge of the exam (e.g., prior exposure to the answer key)? On the flip side, someone who took a very long time to take the assessment  but scored very low would be flagged, too. Might this person have been memorizing questions (i.e., taking exam content to sell to other test takers)?

Who should use it: This report provides valuable test security information for those administering medium/high stakes tests. It can also be used to help determine whether the allotted test taking time window is sufficient for most participants.

How it looks: The scatterplot shows the assessment mean for each participant (X-axis) plotted by the assessment completion time for each participant (Y-axis). Clusters of participants that stand out as having extreme combinations of assessment score and assessment completion time are evident in the scatter plot (e.g., participants with short completion times and high assessment scores would stand out in the lower right section of the graph).

Below  the scatter plot is a table that lists participants that are flagged as having suspicious combinations of assessment score and assessment completion time. This list can be useful in conducting an investigation of what happened in a given context.  Click here to learn more about Questionmark Analytics.

Creating scenario-based assessments

Scenario-based assessments (like this one, for example) are a great way to test learners’ understanding of a specific subject or gauge how someone would react in certain circumstances.

You can create scenario-based assessments in Questionmark Perception version 5 by grouping a series of questions with a single stimulus such as a reading passage, case study, video, image or audio track.

To do this, you would select the appropriate template within the Perception Assessment Wizard to group related questions into a single block. You use text and images to create a static introduction or stimulus that would remain visible in on one half of the window while the questions related to it show up one at a time on the other half.

You can create questions as you would any normal set of questions. Group them in a single sub-topic or place the questions in other relevant topics.

Once your assessment is complete, you can schedule it like any other assessment.

Branching an Assessment to Repeat Itself: Here’s How

There may be times when you want your participants to repeat an assessment – for instance if they fail a quiz or you just want to give them the chance to try an assessment over again regardless of their score. It’s easy to do either of these things in Questionmark Perception.  The ability to branch assessments is particularly helpful if you intend to embed them in a web page like SharePoint, blog or wiki.

Here are tips for allowing participants to repeat an assessment:

Setting up a re-take for participants who do not achieve the required score:

  • When creating or editing the assessment, check the Enable pass / fail check-box in the Assessment Feedback screen of the Assessment wizard and set the required pass % mark.
  • You will also need to select Branching from the Settings menu and indicate that you want the participant to “branch to another assessment.”
  • Just choose the assessment you are currently editing, so that participants will automatically repeat it if they do not achieve the passing score.

Branching an assessment back to itself regardless of the participant’s score:

  • When creating the assessment, ensure that there is only one assessment outcome for any score from 0 to 100%.
  • To do this, uncheck the Enable pass / fail check-box on the Assessment Feedback screen.
  • Follow the same instructions as above to indicate that you want participants to branch to the assessment you are currently editing. That way, they can be taken back to the assessment for another try.

Providing Calculators and Other Tools for Test Takers

There may be times when you want to give test participants access to a certain tool or resource: a calculator or a periodic table, for instance. Or maybe you are giving an open book test about policies and procedures and wish to make a PDF available for participants to refer to.

You can provide these types of resources within your assessments within Perception Authoring Manager. Use the question-by-question (QxQ) template and enable Perception’s Assessment Navigator, which allows participants to move easily from one question to another.

Here are a few rules of thumb for providing tools within assessments:

  • Any tools that you use must be web based or accessible via a network from the computer the participant is using. If you are adding more than one tool, each tool will display in the same order as it appears in the template.
  • The Questionmark Perception version 5 repository comes with a calculator tool that can be enabled or disabled in the template. Other tools can be stored as a resource and added to the repository.
  • You can add third-party tools to assessments, too. These tools are not stored directly in the repository but can be accessed via the Internet or network the participant’s computer is connected to.