Minimizing bias when assessing across culture and language

Student numbers graph

Posted by John Kleeman

I attended a thought-provoking presentation last week by Dr. Janette Ryan of the Teaching International Students project about the rising numbers of international students at universities and the challenges of teaching and assessing them. This inspired me to do some research about the cultural and linguistic challenges involved with assessing in such contexts.

As you will see in the graph on the right, there is an increasing trend for countries to send students for university education overseas, so they can learn from other cultures as well as their own.There are around 3.5 million international students worldwide. The USA is the world’s most popular destination, with the UK and Australia coming second and third.

The UK Teaching International Students project has a page on assessment and feedback, I also found a paper from Oxford Brookes University on Sitting exams in a second language: minimising bias, maximising potential and an Australian guide to Assessing students unfamiliar with assessment practices in Australian higher education.

Here is some advice from these documents. It’s aimed for a university and higher education context, but much here will also be relevant to corporate training.

1. Consider giving extra time for people who are taking an assessment in a language that is foreign to them. You should consider an accommodation in the same way as you would for others who read more slowly, e.g. dyslexic students.

2. Make your questions and instructions clear and unambiguous; use as few words as you need. For someone not working in their native language, each extra word increases cognitive load.

3. Expectations for how to write essays vary between countries and cultures. In some cultures, presenting a contentious statement and asking the student to discuss it is a normal means of assessment; in others this is novel.

4. In some educational settings, the more closely a student replicates the work or words of an expert, the greater the student’s learning or mastery of the subject is considered to be. Elsewhere, replicating the words of someone else is regarded as plagiarism and cheating! Whichever approach you take, tell your students what is expected.

5. Explain well how you are going to run assessments. Styles of teaching, language for grades and ways of assessing vary in different cultures and countries. Your methods may be expected by students from your own country but novel for students from another country.

6. Many international students have a high level of language proficiency but a low level of cultural knowledge. Ensure your assessments do not presume cultural knowledge; using case study questions that make assumptions about prior knowledge or context is a common mistake. The question below is meaningful to Europeans but not to others.

(from Oxford Brookes paper) "As an anthropologist, how would you study Eurovision?"

7. Give plenty of opportunities for students to practice assessments. With software like Questionmark Perception, it’s easy to set up practice tests. This is especially valuable for international students so they can understand what is expected.

8. If your assessment involves participation or work in a group, remember that different cultures have different conventions in group communication — for example about interrupting others or being seen to criticize another in public.

9. Feedback is really important in all assessment. Ensure that it is meaningful and includes any necessary context and doesn’t assume prior knowledge for people who have come from different backgrounds.

10. Above all, set tasks which give all students a chance to succeed

Technology-enabled Learning: Exploring differences worldwide

julie-smallPosted by Julie Chazyn

What makes technology-enabled learning and assessment different in the rest of world? Language instantly springs to mind when we consider what sets one country apart from another. But other differences need to be considered, too, when deciding how best to use technology- enabled learning and assessments.

A recent post in Questionmark CEO Eric Shepherd’s blog explores the differences that arise when you cross social, economic and geographical boundaries.  Eric poses the question: Apart from Language, What Challenges Make Technology Enabled Learning and Assessment Different in the Rest of the World? He then identifies four key points that might drive us to use different kinds of assessments depending on where we are in the world:

  1. Invalid Assumptions About Internet Connectivity (Internet connectivity will create dramatically different experiences for a student in the Amazon and a student in the USA or Europe)
  2. Cost of Internet Device – The cost of purchasing a computer or PDA  in Europe or North America represents a fraction of an average annual salary, whereas in some areas of the world the cost might be 6 – 12 months of an average person’s salary. The resulting use of smaller, lower cost, generally mobile devices in poorer areas of the world calls for the re-sizing of content to accommodate them.
  3. Conformance with Local Laws: Laws regarding data privacy, accessibility and equal access vary from country to country.
  4. Culture: Contrasting value systems can cause different cultures to think differently about such aspects of assessment as cheating. Cheating may be thought of as solidarity within a culture that promotes collectivism and loyalty. 

For more insights on this subject and many others, visit Eric’s blog at http://blog.eric.info.