The CAA Conference, 10 years on

Posted by Steve Lay

I had great fun at the recent CAA (Computer Assisted Assessment) Conference hosted by University of Southampton, UK.  I’d like to thank the team there for taking the lead in organizing the event and opening a new chapter in its history.  This conference builds on the success of the 12 previous CAA conferences hosted at Loughborough University. Although I didn’t go to the first event in 1997, I’ve been a regular attendee on and off for the past 10 years.

I was given the job of summing up the conference and providing closing remarks.  With just two days to read over 30 academic papers I found myself searching for a tool to help me summarize the information quickly.  After a little bit of text processing with a python script and the excellent TagCrowd tool I came up with a tag cloud based on the top 50 terms from the abstracts of the papers presented at the conference:


Assessment obviously remains a key focus of this community, but I was also struck by the very technical language used: system, tools, design, computer and so on.  However, it was also interesting to see which words were missing.  Traditionally I would have expected words like reliability and validity to feature strongly.  Although summative assessment makes an appearance formative assessment does not feature strongly enough to appear in the cloud.  Clearly students, learners and the individual are important but where is adaptivity or personalization?

It is interesting to compare this picture with a similar one taken from the abstracts of the papers in 2000, ten years ago.


An important part of our mission at Questionmark is learning from communities like this one and using the knowledge and best practices to develop our software and services.  During the conference I witnessed a range of presentations covering ideas that we can apply right now through to some fascinating areas of research that point the way to future possibilities.

The conference was a great success, and planning for next year (5th-6th July 2011) has already started.  Check out the CAA Web site for the latest information.

2010 International CAA Conference Q&A

Posted by Mel Lynch

We are looking forward to the 2010 International CAA Conference 20th-21st of July in Southampton on the UK’s South Coast. This  two-day research-led conference hosted by the University of Southampton and organised by its School of Electronics and Computer Science and the Institute of Educational

Lester Gilbert

Technology at the Open University aims to advance the understanding and application of information technology to the assessment process.

I recently spoke with two members of the conference organising committee — Lester Gilbert and Dr Bill Warburton — about this event and some of the trends in e-assessment that will factor prominently in the conference programme. Lester chairs the CAA Conference Local Arrangements Committee and is a Senior Lecturer in IT within the Learning Societies Lab Research Group of the University of Southampton’s School of Electronics and Computer Science.  Bill, the Conference Programme Chair, is CAA Officer at the University of Southampton and a thought-leader on the uptake of technology-enhanced approaches to assessment (as well as a presenter at past Questionmark user events).

Q: What are the major themes of the 2010 CAA conference?

Bill:  A key theme that has already emerged is new ways to enhance student learning through e-assessment. We have papers on topics such as using student contributions to online discussions as assessment tools; the key obstacles to widespread adoption of e-assessment in higher education across the UK; and effective ways to mitigate the emerging threat of remotely administered online exams being compromised by identity fraud.

Lester:  We have sought a wide range of papers relevant to e-assessment and expect some major themes to emerge during the conference. We look forward to them being elucidated in the closing address by Questionmark’s Steve Lay!

Q: Who are some of the speakers who figure prominently in this year’s program – and what will they be talking about?

Dr. Bill Warburton

Bill: The keynote address will be delivered by Dr Paul Brna, Educational Consultant, who will focus on the importance of using e-assessment to provide feedback to learners.

Bobby Elliot will address the fact that student contributions to online discussions are being used increasingly as an assessment tool, but suitable methodologies and standards for their use have been slow to emerge. His “review of rubrics for assessing online discussions” raises some useful points that may inform the directions of future research on the topic.

Nora Mogey will provide an analysis of the key obstacles to the widespread use of summative e-assessment in UK higher education, which addresses the topic of developing institutional strategies for the large scale adoption of online assessment in the future.

Lastly, Kikelomo Apampa’s suggestions for  “Blob-based” Presence Verification may form the basis for innovative and  effective ways to mitigate the emerging threat of remotely administered online exams being compromised by identity fraud.

Q: Why do you think events such as CAA are beneficial for the industry?

Bill:  The CAA Conference has always been an inspirational event where new ideas are forged by delegates into fresh ways of developing assessments practically — and also into new research initiatives.

Lester:  It plays an important role in supporting and facilitating the meeting of ideas and people, keeping up to date with recent developments and being informed about likely future directions.

Q: This will be the first year that the CAA Conference will be held in Southampton. Can you tell me about the new venue and the move?

Lester:  Southampton has gained a reputation as a centre of excellence in e-assessment research and development, particularly in its work on the development of a suite of tools for supporting the IMS QTI interoperability standard and recent reports on e-assessment quality and feedback. As this conference did not take place last year, this year a primary goal is to re-launch the event, reconnect it with its heritage of invaluable papers and re-establish it as an international academic platform for e-assessment research.

*The 2010 International CAA Conference will be held at the DeVere Grand Harbour Hotel in Southampton on the 20th and 21st of July.  For further information and registration visit the conference Web site.