Crowdsourcing in action: a successful experiment at E-Assessment Live

john_smallPosted by John Kleeman

I’m reporting from the E-Assessment Live event at Loughborough University on a practical experience of crowdsourcing assessment content organized by our events team. We had a session with around 20 workstations in a room and gave everyone access via a browser to Questionmark Live, our new software-as-a-service authoring system that allows anyone with a browser to create questions easily and email them out for use in Questionmark Perception.

Most of the people in the room  were not familiar with Questionmark. We asked them all to create a question and email them to me from the system. They all logged into Questionmark Live and wrote a question on their home town which I brought into Questionmark Perception very easily, and within 20 minutes from the first question being authored we had an assessment. See below for a screenshot.


I think the availability of applications like Questionmark Live, which allow easy creation of questions by lots of people at the same time and amalgamation into an assessment, is going to make a big difference in the assessment world. Obvious ideas include getting students to create questions for each other and having SMEs brainstorm and then review questions as a group in an item writing workshop. Essentially harness the power of the crowd by letting each person contribute simultaneously rather than write items sequentially or hierarchically.

I am sure there will be ways of using crowdsourcing for questions that no one has thought of yet and this will hugely improve our productivity. Questionmark Live is free to Questionmark software support plan customers and open for anyone to evaluate. Seeing is believing, so I encourage you to try it out on our website.

Crowdsourcing Content

jim_smallPosted by Jim Farrell

Jay Cross recently mentioned in his Learning Blog a Harvard Business blog post by John Hagel III, John Seely Brown, and Lang Davison about The Collaboration Curve.  I was struck by this statement from their article:

…the more participants–and interactions between those participants–you add to a carefully designed and nurtured environment, the more the rate of performance improvement goes up

This is why Questionmark Live is so important. Think of it as crowdsourcing content from participants who may previously  have been indifferent to the assessment process within your organization. As participation and collaboration increases, the quality of your questions will improve and become far more job-relevant. Look at the following quote from our friends at the Harvard School of Business and replace collaboration curves with Questionmark Live.

Collaboration curves hold the potential to mobilize larger and more diverse groups of participants to innovate and create new value.

Questionmark Live is a vital tool in the creation of a true learnscape.