Conference Close-up: Sustaining large-scale e-assessment
Posted by Joan Phaup
The University of Bradford in the U.K. is delivering four times the number of e-assessments now as it did four years ago – about 60,000 annually these days.
John Dermo from the University’s Centre for Educational Development will tell how this came about – and how the university sustains this high level of assessment – during a case study presentation at the Questionmark European Users Conference in Brussels this October.
John’s session will build on some tips he shared in a previous blog article, but I asked him for a few details about what’s happening at the university and what he’ll be sharing at the conference.
Tell me about your work.
I’m responsible for technology-enhanced formative and summative assessment at the university. I work with a range of people involved in assessment in different ways: academic staff, administrators, IT support, the exams office and the invigilators. As well as initiating changes, I’m the go-between for the different groups.
A couple of projects that took place between 2007 and 2009 have paved the way for expansion and innovation in the area of assessment. Before that, we had limited, ad hoc use of e-assessment, but demand was building up so we built support systems to meet it. We created a workflow model and figured out exactly who did what, and we aimed to make the whole thing scalable. We also built a new e-assessment room to help build up the summative, high-stakes side of our assessment programme.
How has e-assessment at the university changed in the past few years?
With summative assessments, we’ve seen an increase in the speed with which we can get grades to students. Also, it’s now possible to use more multimedia, particularly high-res photographs. The sort of the thing that’s too costly on paper is more practical on the screen. We can also run more authentic types of assessments: we might combine a standard multiple choice assessment with some other online or computer-based tool.
For low-stakes and formative assessments the impact has been slightly different. There has been an increase in the amount of feedback that can be given, and certainly where that’s used it has been very popular with students. There has also been an increase in regular low-stakes assessments, so it has certainly affected the way in which people use blended learning. There’s more interaction now than there was before.
What are the key issues and challenges in achieving sustainable development of e-assessment?
The key things are communication and knowing who does what at what point in the process. It’s easy to think that someone else is going to do a particular task, but that may not be so. Forward planning, fallback plans and communication between the roles is absolutely vital. It’s also important to give staff a certain level of autonomy. Yes, we need processes, but we need to allow for flexibility.
Another thing is keeping training and support as flexible as possible. Some people want to use e-assessment on a regular basis, other people just once or twice a year, so we often need to deliver support and training on a just-in-time basis as well as through more structured programmes. But it’s important to be realistic about what you can do for people. You can’t do everything yourself so you have to set realistic goals and negotiate the most practical way of delivering assessments, managing the workload between different groups as needed.
A big challenge is how to deal with the pioneers who drive innovation. Whilst of course you encourage the pioneers and the innovators, it’s more effective if you weave their enthusiasm into their teams. Relying on just one person won’t make innovation sustainable across the institution. A pioneer might move on, retire, something like that, and where does that leave you? Having innovators share with those around them helps build a sustainable future.
Also, make sure you have some sort of institutionally recognized policy about assessment and keep revisiting it and documenting any changes that you make.
How will your session help people from other institutions expand their use of e-assessment?
Mainly by sharing my experience over the years in an institution where we have seen this growth. I’ll try to draw out practical tips that people can take away. I also want to give people the opportunity to share their own experiences.
I find the Questionmark users community really very, very supportive. It’s one of the reasons I’m attending the conference, in addition to being in constant contact with some members of the community. I think the more we can share our experiences the better.
What are you looking forward to at the conference?
A lot! I’m looking forward to meeting up with some people I haven’t seen in a couple of years. I’m also very interested in the new functionality in Questionmark Perception version 5 because we are in the process of upgrading. And I want to learn about integrating v5 with other tools, in particular virtual learning environments.
There’s still time to register for the conference. Click here to learn more.