Find the gaps, then design the learning

Posted by Jim Farrell

Having spent my entire career (amazingly coming up on 20 years now) in the learning industry, I find it curious that I often cannot find time for my own learning.  I always have a healthy stack of items on my nightstand that I have every intention of reading, but the stack is rarely diminished.

Twitter has become my digital nightstand. Instead of weeding through articles and websites, twitter has helped me “hire” my own personal curators to show me what is new and exciting in our industry. One recommendation from one of my curators, @cammybean, was the book, Design for How People Learn, by Julie Dirksen. Now the title alone is enough to grab most professional educators, but what really got me was @cammybean saying that most of the illustrations were stick figures. I am a huge fan of stick figures in illustrating learning so I was sold.

According to Julie, before you can create a good learning experience, you have to understand the gaps: knowledge, skills, motivation, environment and/or communication. I use and/or because it is likely that there are multiple gaps that must be addressed when dealing with more than one learner. How do we find gaps? We ask questions!

In my experience, simple surveys or assessments can help uncover most types of gaps. For example, letting participants rate the knowledge level on the topics that are to be covered can give a lot of insight about the class make-up. And if you really want to check their knowledge, ask them questions about things they say they have medium understanding of while avoiding topics about which they say they have minimal knowledge. Think about it: Why would you ask questions when the participant has already told you they don’t know the answer?

Asking participants how often they perform certain skills and how important they are can give you information about how much opportunity they will have to practice those skills back at work. It will also help you gauge how motivated they are to learn them. It’s also great to ask learners is if they know why they are attending the training. This can uncover communication gaps that could hinder their success.

We all know that evaluating participants’ progress throughout the learning process is valuable, but asking questions to determine the gaps participants are bringing to learning can make or break a training experience.

The CAA Conference, 10 years on

Posted by Steve Lay

I had great fun at the recent CAA (Computer Assisted Assessment) Conference hosted by University of Southampton, UK.  I’d like to thank the team there for taking the lead in organizing the event and opening a new chapter in its history.  This conference builds on the success of the 12 previous CAA conferences hosted at Loughborough University. Although I didn’t go to the first event in 1997, I’ve been a regular attendee on and off for the past 10 years.

I was given the job of summing up the conference and providing closing remarks.  With just two days to read over 30 academic papers I found myself searching for a tool to help me summarize the information quickly.  After a little bit of text processing with a python script and the excellent TagCrowd tool I came up with a tag cloud based on the top 50 terms from the abstracts of the papers presented at the conference:


Assessment obviously remains a key focus of this community, but I was also struck by the very technical language used: system, tools, design, computer and so on.  However, it was also interesting to see which words were missing.  Traditionally I would have expected words like reliability and validity to feature strongly.  Although summative assessment makes an appearance formative assessment does not feature strongly enough to appear in the cloud.  Clearly students, learners and the individual are important but where is adaptivity or personalization?

It is interesting to compare this picture with a similar one taken from the abstracts of the papers in 2000, ten years ago.


An important part of our mission at Questionmark is learning from communities like this one and using the knowledge and best practices to develop our software and services.  During the conference I witnessed a range of presentations covering ideas that we can apply right now through to some fascinating areas of research that point the way to future possibilities.

The conference was a great success, and planning for next year (5th-6th July 2011) has already started.  Check out the CAA Web site for the latest information.