Five tips for enhancing test security using technology

Headshot JuliePosted by Julie Delazyn

Test security is a topic that comes up time and time again on education and company forums.  You can improve test security by changing the physical test-taking environment, but you can also use technology to tackle certain security issues.

Here are are five tips that can help you use technology to address security challenges:

  1. Randomize: Shuffling the order of the choices can help protect the security of the assessment. The questions can also be delivered in a random order themselves — to help prevent cribbing when users are sitting in non-screened assessment centers.
  2. Encrypt: With so many tests and exams being delivered via the Internet or an intranet, encryption can protect against interception. A Secure Socket layer (SSL) is a protocol that allows the browser and web server to encrypt their communication; anyone intercepting the communication can’t read it.
  3. Schedule: You can discourage cheating by specifying user names and passwords, setting assessment start times, limiting the length of time for an assessment and the number of times it may be taken.
  4. Monitor: A participant can’t start a monitored assessment until a proctor or invigilator has logged on to verify the participant’s identity. The monitor can be limited to a range of IP addresses to ensure that a certain physical location is used to administer the assessment.
  5. Secure browsers: It is possible to ‘lock down’ computers to keep participants from accessing other applications and websites while taking a medium- or high-stakes assessment. A secure browser prevents candidates from printing, capturing screens, accidentally exiting the assessment viewing source, task switching, etc.

Want more info? Download the White Paper: Delivering Assessments Safely and Securely  [registration required]

11 Tips to help prevent cheating and ensure test security

Headshot JuliePosted by Julie Delazyn

With the summer behind us, it’s officially fall, and that means schools, colleges and universities have launched into a new academic year.

In this time of tests and exams, the security of test results is crucial to the validity of test scores. Today, I’d like to introduce 11 tips to help prevent cheating and ensure assessment security.

1. Screening tests — A small pre-screening can be administered to prevent people from taking an assessment for which they are not yet prepared.

2. Candidate agreements — Candidate agreements or examination honor codes are codes of conduct that a participant must agree to before they start an assessment . Candidate agreements generally are phrased in a personal manner ; the participant agrees by clicking on an ―OK‖ or ―Yes‖ button to the code of conduct for the exam

3. Limiting content exposure/leakage — In order to limit the amount of question content being shown to a participant at any given time, consider using question-by-question templates. These present questions one at a time to participants so that exam content is not completely exposed on screen.

4. Screening participants who achieve perfect scores — Many organizations will automatically investigate participants who achieve perfect scores on an assessment. Perfect scores are rare events, and could be attributed to a test-taker having had access to answer keys. The Questionmark Score List Report provides a fast and easy way to identify participants who obtain 100% on their assessments. An organization can then conduct an investigation of these participants to ensure that no suspicious behavior had occurred.

5. Verifying expected IP addresses — If assessments are to be taken from a specific location, often the IP address of the computer in that location will be known. Verifying expected IP addresses is a useful way to screen whether participants somehow took an assessment from an unauthorized location.

6. Reviewing time to finish information — The overall time it takes for a participant to complete an assessment can be a useful way to screen for suspicious behavior. If a participant takes a very short time to complete an assessment yet achieves a high score, this could be an indication that they cheated in some way.

7. Using Trojan horse or stealth items — Trojan horse or stealth items can be used to help detect whether a participant has memorized the answer key. Stealth items are inserted into an assessment and look just like the other questions, but they are purposely keyed incorrectly and one of the distracters is marked as the correct answer.

8. Post information that cheater prevention tactics are used — Inform participants that cheater -detection tactics are regularly employed. This can help to deter the low – motivation cheaters.

9. Proper seating arrangements for participants — Implementing a seating plan where participants are equally spaced, with limited ability to see another participant‘s screen/paper, is an import strategy.

10. Using unique make-up exams — When offering a make-up exam, make sure to administer it in the same strict proctored environment as the scheduled exam. Also, having another test form available specifically for make-up exams can lessen the risks of cheating and exposure for the actual large-scale exam.

11. Using more constructed response questions — Constructed response questions, like essay or short answer questions, provide less opportunity for participants to cheat because they require them to produce unique answers to questions.

If you’d like more details about these and other tips on ensuring the security and defensibility of your assessments you can download our white paper, “ Delivering Assessments Safely and Securely.”

Candidate Agreements: Establishing honor codes for test takers

julie-smallPosted by Julie Delazyn

With schools, colleges and universities now fully launched into a new academic year, it’s certain testing season!

The security of test results is crucial to the validity of test scores – something we explored in a previous post. Today, I’d like to look at another helpful tool for promoting secure and fair tests: the candidate agreement or examination honor code.

These agreements outline what is expected of test takers. They present a code of conduct that test takers must agree to before they start an assessment. This can be done manually as an outline or electronically before an online exam begins. When participants sign the code, they’re consciously acknowledging the rules and the repercussions of cheating. Such codes apply to all types of high-stakes testing, such as certification tests.

What expectations should you include in a candidate agreement? Here are some to consider:

  • The candidate must abide by the rules of the test center, organization, or program
  • The candidate will not provide false ID or false papers
  • The candidate cannot take the test on behalf of someone else
  • The candidate will not engage in cheating in any form
  • The candidate will not help others cheat
  • The candidate will not use aids that are not allowed
  • The candidate will not solicit someone else to take the test
  • The candidate will not cause a disturbance in the testing center
  • The candidate will not tamper with the test center in any way
  • The candidate will not share information about the assessment content they saw (non-disclosure agreement)
  • The test vendor will have the option to terminate the assessment if suspicious behavior is detected

If you’d like more details about these and other tips on ensuring the security and defensibility of your assessments you can download our white paper, “Delivering Assessments Safely and Securely.”

Effectively Communicating the Measurement of Constructs to Stakeholders

greg_pope-150x1502

Posted by Greg Pope

I co-wrote this article Kerry Eades, Assessment Specialist, Oklahoma Department of Career and Technology Education, a Questionmark user who shares my interest in test security and many other topics related to online assessment.

Kerry Eades

There are many mentions on websites, blogs, YouTube, etc. about people (employees, students, educators, school administrators, etc.) cheating on tests. Cheating has always been an issue, but the last decade of increased certifications and high-stakes testing seems to have brought about a significant increase in cheating. As a result, some pundits now believe we should redefine cheating and that texting for help, accessing the Web, or using any Web 2.0 resources should be allowed during testing. The basic idea is that a student should no longer be required to learn “facts” that can be easily located on the internet and that instruction should shift to only teaching and testing conceptual content.

There are many reasons for testing (educational, professional certification and licensure, legislative, psychological, etc.) and the pressures that stakeholders feel to succeed at all costs by “teaching to the test” or to condone any form of cheating is obviously immense. Those of us in the testing industry should, to the best of our ability, educate stakeholders on the purpose of tests and on the development and measurement of constructs. Having better informed stakeholders would lessen the “need” and “excuses” for cheating and improve the testing environment for all concerned. A key element of this is promoting an understanding of how to match the testing environment to the nature of an assessment: it is appropriate to allow “open book” assessments in some cases but certainly not all. We must keep in mind that education, in general, builds upon itself over time, and for that reason, constructs must be assessed in a valid, reliable and appropriate manner.

Tests are usually developed to make a point-in-time decision about the knowledge, ability, or skills of an individual based upon a set of predetermined standards/objectives/measures. The “value” of any test is not only this “point-in-time” reference, but what it entails for the future. Although examinees may have passed an assessment they may still have areas of relative weakness that should be remediated in order for them to maximize their full potential as students or employees. Instructors should also observe how all their students are performing on tests in order to identify their own instructional weaknesses. For example, does the curriculum match up with the specified standards and the high level of thinking in those standards? This information can also be aggregated and analyzed at the local, district, or state level to determine program strengths or weaknesses. In order to use scores in a valid way to make decisions about students or programs, we must begin by clearly defining and measuring the psychological/educational constructs or traits that a test purports to measure.

Measuring a construct is certainly complex, but what it boils down to is ensuring that the construct is being measuring in a valid way and then reporting/communicating that process to stakeholders.  For example, if the construct we are trying to measure in an assessment is “Surgery Procedure” and if the candidate passes the test, we expect that the person can recall this information from memory where and when needed.  It wouldn’t be valid to let the participant look up where the liver is located on the Internet during the assessment, because they would not be able to use the Internet while they are halfway through a surgical procedure.

Another example would be “Crane Operation” knowledge and skills.  If this is the construct being measured and it is expected that candidates who pass the test can operate a crane properly, when and where they need to, then allowing them to tweet or text during their crane certification exam would not be a valid thing to do (it would invalidate the test scores) because they would not be able to do this in real life.

However, if the assessment is a low stakes quiz that is measuring the construct, “Tourist Hot Spots of Arkansas,” and the purpose of the quiz is to help people remember some good tourist places in Arkansas, then an “open book” or an “open source” format where the examinee can search the internet or use Web 2.0 resources is fine.

Effectively communicating the purpose of an assessment and the constructs being measured by it is essential  for reducing the instances of cheating. This important communication  can also help prevent cheating from being “redefined” to the detriment of test security.

For more information on assessment security issues and best practices, check out the Questionmark White Paper: “Delivering Assessments Safely and Securely.”

4 Tips to Help Ensure the Security of Intellectual Property

julie-smallPosted by Julie Chazyn

Protecting the intellectual property contained in a test or exam is essential, not only because of the time, effort and cost of creating assessments but also because IP theft undermines the accurate measurement of knowledge and skills.

Protecting intellectual property protects the credibility of tests. Here are four tips for helping to ensure the security of intellectual property:

Create and administer multiple test forms

Rather than having only one form of the assessment being administered, delivering multiple forms of the same exam can help limit item exposure. This method also allows for the possibility of interspersing large-scale integrated beta test questions within the forms to collect psychometric information on newly developed questions.

Restrict and control administration of beta test items

Beta testing questions is an important part of high-stakes assessment, ensuring the psychometric quality of questions before they appear on actual assessments. However, it is vital that a well conceptualized beta test model is in effect to limit the exposure of newly developed questions to participants.

Update exam forms periodically

Letting exam forms become stale can over-expose questions to participants, increasing the likelihood of IP theft. An organization could consider retiring old exam forms and turning them into exam prep materials that can be sold to participants. In this way, participants could periodically expect new practice questions.

Produce exam prep materials

Organizations should consider making exam prep materials available to participants before an assessment. This will help reduce the demand for participants to try to obtain exam questions via illegal means as they will have access to the type of questions that will be asked on the actual assessment.

For more details on this subject, plust information about various means for deploying a wide range of assessment types with assurance, download our White Paper: Delivering Assessments Safely and Securely.

Delivering Assessments Securely: What delivery environment is best?

Joan PhaupPosted by Joan Phaup

In a previous post I mentioned that the stakes of an assessment should drive decisions about delivery and security requirements.

Today I’d like to share this chart that can help you determine what delivery environment would be suitable for various types of assessments. The chart appears in our white paper, Delivering Assessments Safely and Securely, which offers many more details on this subject–the aim being to help you avoid incurring unnecessary costs while at the same time giving your assessments the security levels they require.

delivery method