Tips for preventing cheating and ensuring assessment security

julie-smallPosted by Julie Chazyn

Last week I wrote about tips for protecting  intellectual property.  It’s equally important keep people from cheating on tests, so here are three pointers on that subject.  I’ll be following this up with more tips in future posts.  Leave me your comments; we can always add to lists like these!

.Screening tests

Consider givinig a small pre-screening test to prevent people from taking an assessment that is beyond their current ability level. If a participant can‘t answer a certain number of these questions correctly they will not be allowed to see the remainder of the assessment. When the time does come for them to take the test,  they will not have already seen its content.

Candidate agreements

Candidate agreements or examination honor codes require  a participant’s agreement before they start an assessment, say by clicking on an “OK “ or  “Yes” button after reading the exam’s code of conduct.

The code might say something like this:  “I agree to answer the questions on this assessment without obtaining assistance from another person or via electronic means. I agree to not to share my answers with anyone during or after the exam. I further agree to not memorize or otherwise steal the intellectual property contained in this exam. I accept that if any of these conditions are violated, my exam results will be set to a zero, I will not be able to retake the exam for a period of 10 years, and I may be charged with a crime under regional laws.”

Here are some topics you might want to cover in a candidate agreement:

  • The test vendor will have the option to terminate the assessment if suspicious behavior is detected
  • The candidate must abide by the rules of the test center, organization, or program
  • The candidate will not provide false ID or false papers
  • The candidate cannot take the test on behalf of someone else
  • The candidate will not engage in cheating in any form
  • The candidate will not help others cheat by disclosing information about the assessment
  • The candidate will not use aids that are not allowed
  • The candidate will not solicit someone else to take the test
  • The candidate will not cause a disturbance in the testing center
  • The candidate will not tamper with the test center in any way
  • The candidate will not share information

Limiting content exposure/leakage

In order to limit the amount of question content being shown to a participant at any given time, think about using question-by-question templates. These present questions one at a time to participants so that exam content is not completely exposed on screen. Participants who may intend to take pictures of the exam content or otherwise steal intellectual property will not be able to do so all at once.

There are many fine resources for learning how to prevent cheating. Two of thes are books by Dr. Gregory Cizek:  Cheating on Tests: How to Do It, Detect It and Prevent It and Detecting and Preventing Classroom Cheating: Promoting Integrity in Assessment.

There’s also our white paper: “Delivering  Assessments Safely and Securely,” and of course this blog! Watch for more security tips in my future posts.

Item Analysis Analytics Part 5: Outcome Discrimination and Outcome Correlation

greg_pope-150x1502

Posted by Greg Pope

In my previous blog post I dived into some details of item analysis, looking at example questions and how to use the Questionmark Perception Item Analysis Report in an applied context. I thought it might be useful in this post to talk about outcome discrimination and outcome correlation, as people sometimes ask me how are these different or the same, when should I use one or the other, and so on. The fact of the matter is that you can use one or the other and often it comes down to preference as they both yield quite similar results.

Outcome discrimination is the proportion of the top (27% according to assessment score) of participants who selected a response option minus the lowest (27% according to assessment score) of participants who selected each response option to the question. What you would expect is that participants with the highest assessment scores should select the correct response option more often than participants with the lowest assessment scores. Similarly, participants with the highest assessment scores should select the incorrect distracters less often compared to the participants with the lowest assessment scores.

Outcome correlation is a point-biserial correlation that correlates the outcomes scores that participants achieve to the assessment scores that they achieve. So rather than comparing only the top and bottom 27% of participants, the outcome correlation looks at all participants using a standard correlation approach.

If you are thinking that outcome discrimination and outcome correlation sound like they might be related to one another, you are right! High outcome discrimination statistics generally will result in high outcome correlations. In other words, outcome discrimination and outcome correlation statistics are highly correlated with one another. How correlated are they? Well, I looked at many real-life questions from Item Analysis Reports that customers have shared with me and found a positive correlation of 0.962, which is really high.

part-5

In my next post I will provide some criteria that can be used for outcome discrimination and outcome correlation coefficients to judge whether a question is meeting the grade in terms of psychometric quality.