The University of Bradford currently delivers around 60,000 assessments per academic year with Questionmark Perception, four times as many as four years ago, as you can see in the graph below.
I spoke to John Dermo from the Centre for Educational Development at the University about advice he would give others who are seeking to expand their use of e-assessment and how to sustain this over time. John explained that a common pattern of use at Bradford is that an instructor comes to the Centre interested in delivering a summative exam online, with mainly interested in saving time by having the computer do the scoring. In many cases they then start seeing the benefits of formative assessment, firstly by using practice tests to prepare for the exam, and then by adding topic and specific question feedback to make practice tests into formative tests that help students learn. So at Bradford, the initial drive for e-assessment is often to reduce grading effort, but once people have experienced the benefits there, they often want to do formative assessments too.
Here are ten tips from John on making your e-assessment policy sustainable:
1. Building confidence is vital. Confidence comes from training and support, which needs to be sustainable and flexible. In the early stages, the Centre made sure that technical support was visible in the room during every exam in case of problems. Now with e-assessment more established, they have someone support at the beginning and end only. Invest in lots of support to begin with.
2. Give students practice tests ahead of summative exams. Students will feel more comfortable and make fewer mistakes in exams if they have a chance to practice in the format beforehand. Bradford also provides training videos to help students become familiar with the software.
3. Engage with course teams rather than just individuals. Often it’s an individual who is an innovator and does the first set of e-assessments, but aim to work with their team not just them, as that is more sustainable in the long term. The dip you can see in the graph above in 2009-10 was due to a couple of heavy pioneering users leaving the University or going on sabbatical. Individuals often lead the way, but engage with course teams to make the use of e-assessment sustainable.
4. Be flexible on environment. Bradford has a 100-seat specialist e-assessment suite, but online assessment also takes place elsewhere (e.g. overflow rooms for large modules, other locations for students with accessibility needs, also for distance and overseas students), so make your procedures flexible.
5. Distinguish summative and formative assessments. For summative assessments, schedule and manage them centrally to ensure the right level of support and security. But for formative assessments, empower course teams to make them happen as independently and easily as possible from within the University VLE (at Bradford, they’ve found the Blackboard Connector to Questionmark Perception makes this very easy).
6. Define a formal policy for e-assessment and have it approved at institutional level. This will ensure that all stakeholders concerns are addressed (e.g. accreditation, IT, student records, disability office). Bradford’s current policy is here. Remember that your policy needs to be reviewed at regular intervals.
7. Include in the policy quality assurance procedures for summative assessments. Unlike in a paper exam, e-assessment lets you change a question at the last minute, but to retain quality, you need to ensure that formal policies on review are followed.
8. Define roles and responsibilities clearly. Moving from paper to electronic assessment can introduce ambiguities as to who is responsible for what, so have the conversations to resolve these. You want to make sure that no important steps are allowed to “fall between the gaps”, where everybody assumes that someone else is responsible.
9. Involve peer institutions as a sounding board. For instance, Bradford has learned from Dundee University’s experiences with CAA policies and from talking to many other institutions about the use of Questionmark Perception. There is a healthy Questionmark Perception user community within the HE sector, which communicates via the Questionmark Users Conference, other e-learning and e-assessment conferences and via various online communities.
10. Success means empowering course teams to create reliable, valid and practical e-assessments. Support, help and train them to do so, working with the departments to meet their teaching and learning needs, and to deliver the institution’s teaching and learning strategy.
For more information on how the University of Bradford uses Questionmark Perception, see the University’s internal best practice document.