Using the Demographic Report

The Demographic Report was recently added to Questionmark Analytics. Here are the basics:

What it does: The demographic report breaks down results by demographic variables such as language,  course name, location, department, instructor, and job role or military rank. This report can recognize up to 10 demographic variables recorded by an  assessment. Users can review the performance of assessment data broken out by a demographic variable, using data stored in special fields within the database.

Who should use it: Assessment, learning and education professionals can use this report to zero in on test results according to specific demographic information.

How it looks: Graph 1 in this example shows the assessment mean score for the selected demographic. It displays the average percentage score achieved as well as the high and low results.  Graph 2 shows the number of results for the selected demographic and includes an overall calculation of the number of results found.

You can assign several filters to limit the information included in the report:

  • Assessment (mandatory)
  • Group
  • Date
  • Attempt
  • Special field

Using the Assessment Completion Time Report

The Assessment Completion Time Report is one of three new reports in Questionmark Analytics. Here are the basics:

What it does: The assessment completion time report graphically monitors how long, on average, it took each participant to complete an assessment in relation to  his or her overall score.  For example, someone who took very  little time to complete the assessment  but achieved a very high score would be flagged. Might this person have had prior knowledge of the exam (e.g., prior exposure to the answer key)? On the flip side, someone who took a very long time to take the assessment  but scored very low would be flagged, too. Might this person have been memorizing questions (i.e., taking exam content to sell to other test takers)?

Who should use it: This report provides valuable test security information for those administering medium/high stakes tests. It can also be used to help determine whether the allotted test taking time window is sufficient for most participants.

How it looks: The scatterplot shows the assessment mean for each participant (X-axis) plotted by the assessment completion time for each participant (Y-axis). Clusters of participants that stand out as having extreme combinations of assessment score and assessment completion time are evident in the scatter plot (e.g., participants with short completion times and high assessment scores would stand out in the lower right section of the graph).

Below  the scatter plot is a table that lists participants that are flagged as having suspicious combinations of assessment score and assessment completion time. This list can be useful in conducting an investigation of what happened in a given context.  Click here to learn more about Questionmark Analytics.