Exams and social media: is it really spying?

Steve Lay HeadshotPosted by Steve Lay

While I was traveling back from our US Users Conference several weeks ago, a debate was raging on social media following news that a testing company had been monitoring Twitter to detect evidence of leaked content. The Guardian newspaper, for example, reported that a New Jersey superintendent had found this ‘disturbing’.

In case you haven’t read about this case, here are the basics: after school, a student tweeted information about a test administered earlier that day. An automated Web monitoring system discovered the tweet, and the school was notified. The student later deleted the offending tweet.

According to the test provider, administrators are supposed to tell participants that sharing any test question online is prohibited. It isn’t clear from the press reports whether this warning was issued prior to the test or whether the student would have considered the tweet prohibited or not. Whatever the case may be, enough information was shared to trigger the automated warning.

Perhaps more interesting than the story itself is the reaction to it. Strong words have been used, but should monitoring social media really be regarded as spying?

The monitoring of online forums to check for exam leaks is not new. It goes back to the very earliest days of the Internet. When I first read about this case my first reaction was that this type of thing is happening all the time. Indeed, brand owners are constantly monitoring social media to help them understand the public’s reaction to their products and services and to help them target their advertising more effectively. Copyright owners also monitor the web to check for infringement. Trademark owners must pro-actively monitor for misuse to prevent their trademarks from becoming unenforceable. So if an organization has such rights, wouldn’t monitoring the web–including social media–to enforce them surely be expected?

This assumption is probably naive. Many people are not aware that this information is available in a form that can be subscribed to. They do not understand the subtle difference between a comment being made in a ‘public place’ like twitter and it being instantly discoverable. In our everyday experience, a conversation that happens in a public place like a café or store is not recorded, transcribed and then made instantly available to business partners of the venue. In this case, the student, the student’s parents and even the superintendent were surprised and shocked by the level of surveillance. They reacted as if a private conversation had been overheard.

It is interesting to contrast this recent case with one reported by Techcrunch in 2009, when information from Facebook was used to hold students to account for cheating. But in the Facebook case, the information was discovered by other students and brought to the attention of the test authorities. Why would the students do that? Likely because test takers are key stakeholders too! If cheating becomes commonplace, then the test will become worthless. So both the test publisher and the test taker have an interest in ensuring fair practice.

Coming back to the rogue tweet, what’s frustrating here is that there is no suggestion that the test taker was trying to cheat or to help someone else cheat. I haven’t seen the 140 characters in question, but it seems likely that the tweet was just a trivial extension of the type of verbal conversation that people frequently have after taking tests.

The mismatch in privacy expectations and the feeling that the student was being accused of malpractice were a toxic mix. Both of these can be avoided.

When monitoring people using CCTV or similar technologies, it is good practice to inform people that they are being monitored, and for what purpose. In many jurisdictions this may also be a legal requirement. Likewise, why not inform test takers of the type of monitoring that is taking place and why? This may have the added advantage of helping to inform them about the risks to their own privacy that over-sharing on social media can pose.

Also, when issues are flagged by monitoring services, test publishers should think carefully about any follow-up actions. Are these actions consistent with the stated purpose of the monitoring? Are they proportionate? Remember, the test taker and the test publisher should be on the same side!

3 Responses to “Exams and social media: is it really spying?”

  1. Bjorn says:

    This kind of monitoring, and the way it was handled, is way past the “creepy” line. To do this kind of monitoring they have to store large amounts of data, literally keeping a file on every student. On children!? Think about it, they didn’t even do this themselves, but hired a third party to do the data collection. A company that absorbs the liability and makes its money by how many “cheaters” it finds – giving them financial motivation to use the most invasive means that it can get away with. Maybe people in the UK have gotten used to being surveilled, but it really isn’t normal. Where does it stop? Would you have a problem with the dairy industry making refrigerator manufacturers include cameras in your fridge – so they can monitor that you don’t use “their” milk after the use-by date?

  2. Steve Lay says:

    Thank you for your comments. You make an excellent point about the data being generated by children (see below), but I feel the criticism with respect to storage is misdirected.

    The information is stored and made available to anyone prepared to pay for it by the social media company. In this case, Twitter. I don’t know how this particular piece of monitoring software works but you can do a simple experiment yourself. If you open your favourite Twitter client and search for a phrase like “Henry V” you’ll find a relatively small number of tweets, perhaps 2-3 per hour. Now imagine that you’ve published an exam with a question about the Shakespeare play “Henry V”. You can redo your search in the hours following the exam to see who has suddenly started talking about it. You don’t need to store information for any length of time, it is enough to just monitor the stream of new tweets.

    The point I make in my blog post is that the data collection and, more importantly, distribution is already being done by the social media companies. This type of surveillance has become normal whether we like or not, but it has got way ahead of the public understanding. In my opinion, this is just as true in the UK as it is elsewhere. It is worth comparing this case with the recent investigation into a UK politician for the sharing of an exit poll result via Twitter before voting had finished: http://www.bbc.co.uk/news/uk-england-leeds-32638176

    You could argue that adults should be able to understand the difference between a private email and a public tweet but your point about children is well made. In the case of Twitter they “do not knowingly collect personal information from children under 13”, well below the age at which we would consider someone an adult. My blog post highlighted the headache that oversharing causes the owners of confidential exam questions but as your response suggests, the privacy implications of oversharing raise far wider questions. I couldn’t agree more.

  3. Bjorn says:

    I think you might be missing a key point in my comment. Just because something is possible doesn’t mean that it is a good idea. That you achieve some positive goal for yourself doesn’t outweigh all other concerns. Recent news notwithstanding, in certain parts of the US there are laws against police officers shooting at criminals that run away from the police if the crime they’re suspected of isn’t a death-penalty offense; and police can be prohibited by policy from initiating high-speed car chases when there was no violent crime – since the high danger to the general public outweigh catching the criminal.

    Just because Twitter publishes a feed, doesn’t mean that it is a good idea (or even legal in some places) to join this data stream (database?) with other databases containing information about people – and especially children.

    And then there is the creepy factor.. In general, if you think something is creepy, adding “on social media” doesn’t make it less creepy: If we go back to the time before social media, kids might have been hanging out on playgrounds, in skate parks, the local football field, etc. All of those are public places. You, as the author of an exam with a question about “Henry V”, could lurk around these places to catch anyone talking about the exam. But since you can only be in one place at a time, perhaps you hire a company that has a fleet of unmarked vans with more sophisticated listening equipment to monitor all/most of these public places.

    This kind of monitoring, by yourself or your hired company, will undoubtedly catch much more information than just whether “Henry V” was mentioned. You’ll have to store the info if you’re going to use it for any disciplinary action. And really, why would you ever delete it? You could data-mine it to discover how to be a more efficient lurker, or you could sell it to another company to do this for you – or companies that sell this analysis as a service will buy it from you and sell it to someone else. Move forward ten years and these children will start their first job, enter politics, etc. while you have a company sitting on all the silly things they did when they were children. We could be living in a society run by people who never pushed the limits when they were kids.

    Surveillance is a sledge-hammer that ought to only be used in a strictly regulated and time-limited fashion, when the goals far outweigh the negative consequences. Questions about “Henry V” don’t even register on this scale. When writing exam questions, it might be worth appreciate that talking about a hard exam is natural (and even healthy) for students. Perhaps exam questions need to be considered like “fresh produce” to feed to a living/changing exam, rather than question “bricks” that build an unchanging exam “cathedral”?

Leave a Reply