Crab cakes, customers and cool code in Charm City

John Kleeman HeadshotPosted by John KleemanHarbor View

I’ve just back from the Questionmark user conference in Baltimore “Charm City”, famous for its delicious crab cakes.

The best crab cakes let the flavour of crab shine through the other ingredients that hold the cake together. Similarly, with a conference, the main ingredient is the learning we take home with us. The other things that hold the event together – the networking, social events and discussions that take place – make the learning very enjoyable.

Here are six trends I observed last week:

1. Scaling up works well and is commonplace. Many Questionmark users have high volumes without drama. One case study at the conference told of a company deploying Questionmark with SAP in two months to deliver safety qualification assessments to 35,000 employees. And another told of delivering 2.3 million Questionmark assessments in 2012.

2. Authoring improvements. Our recent web-based authoring improvements are exciting users. In the 25 years we’ve been operating, Questionmark software has always been about enabling users to easily create, deliver and report on tests, quizzes, surveys and exams. There is a lot of excitement about what we’ve introduced recently in web-based authoring of questions and assessments, especially the way people can easily collaborate and see and review changes in items.

3. Acceptance of the Cloud. Some organizations prefer to deploy on-premise, others deploy in the Cloud with Questionmark OnDemand, and we support both. With other HR systems moving to the Cloud, more Questionmark customers are looking at moving to OnDemand. One influence is that SAP is telling all its HR customers that the future for them is in the Cloud with SuccessFactors; people are thinking “if you’re using SAP in the Cloud, then it makes sense to use Questionmark there, too”.charles networking

4. 70:20:10 is a given, and people are learning how to use it. Charles Jennings keynote was not without its controversies – many people still believe in the Kirkpatrick and Phillips models of learning evaluation, but he suggests levels 1, 2 and 5 have dubious validity in today’s learning environment. But there is widespread acceptance that the 70:20:10 model makes sense in corporate learning – that we learn 70% of what we learn on the job, 20% from others and only 10% from formal learning. There are questions around how you assess workplace learning – observational assessments are many people’s answer, and how you combine 70:20:10 with mandatory compliance training and ensuring people learn to work safely.

5. New methods of secure delivery matter. The availability of Questionmark Secure for use with Macs is making it easier for organizations, especially universities/colleges, to use a secure browser for delivering exams. People are also starting to use remote proctoring via organizations like our partner, ProctorU. Mobile assessment delivery is also now real not just being experimented with.

6. Cool code makes a difference. There’s been a lot of talk in recent years about how HTML 5 will make a difference to user interfaces and it’s great to see this happening for real. Questionmark is using HTML 5 to make our authoring and delivery interfaces easier and more natural. See for example this video: How to create hotspot questions for quizzes and tests. Code like this is exciting our customers and is going to provide very compelling web interfaces moving forwards.

I hope you enjoyed this and our other blog entries about the conference. We do our best to share about Questionmark and assessment remotely, but if you have a chance to attend one of our US or European user conferences in person, I promise you will learn and gain value.

Tiger sharks, dolphins and Texas – oh my!

Headshot JuliePosted by Julie Delazynfish

As we wrap up our annual Questionmark Users Conference, many customers have commented that with so many different types of presentations and discussions to choose from, it’s best to bring colleagues and use the “divide and conquer” method to cover all the bases. It’s been great to see organizations sending a range of employees so that everyone can go to a different kind of session and then brief each other on what they learned.

Yesterday’s keynote address by Charles Jennings gave us a lot to thing about regarding the 70:20:10 framework and the challenge of measuring informal and workplace learning. Incidentally, his column in yesterday’s Training Industry Quarterly picked up the theme of “Extracting Learning from Work,” – something Charles touched on during his talk.

jelliesLive blogs from both the keynote address and the opening general session will give you a taste of what we’ve been hearing about during the last couple of days. And we hope you will dive into the conversation on Twitter, using the #qmcon tag.

Last night we celebrated Questionmark’s 25 year anniversary at the National Aquarium, just across Baltimore’s Inner Harbor from the conference hotel. We had the Aquarium all to ourselves and enjoyed a dolphin show before dinner and birthday cake.

If you were not able to make it to the conference this year, stay in touch with us on Twitter, Facebook and of course, this Blog. And mark your calendar for March 4 – 7, 2014, when we will meet at the Grand Hyatt in San Antonio, Texas, for the Questionmark 2014 Users Conference!

Look out for pictures on our Flickr page from this intense and always-fun learning event.

Podcast: Charles Jennings on measuring informal and workplace learning

Posted by Joan Phaup

I’m eagerly looking forward to the keynote presentation Charles Jennings will deliver at the Questionmark 2013 Users Conference  on The Challenge of Measuring Informal and Workplace Learning.

Charles Jennings

Charles is one of the world’s leading thinkers and practitioners in learning and development — currently head of Duntroon Associates and previously Chief Learning Officer for Reuters and Thomson Reuters.He will be talking at the conference about how the 70:20:10 learning framework — based on studies that show high performers learn approximately 70% from experience, 20% from others and 10% from formal study – is being adopted by many organizations around the world.

The keynote will address how this framework serves as a strategy for extending development beyond formal, structured learning to include informal and experiential learning opportunities.

I spoke with Charles recently and asked him for some details about his presentation. For example:

  • How would you describe the 70:20:10 framework?
  • What are the key challenges of measurement and evaluation within that framework?
  • How will your conference presentation address those kinds of challenge?
  • What advice would you give to organizations that want to use online assessments to measure the effectiveness of informal and experiential learning?

If you’d like to find out how he answered, listen to this podcast or read the transcript. There will be much, much more, of course, in his keynote address and in the conference program, which we are busy planning right now.

Early-bird registration savings are available through November 16 — so keep an eye on the conference website and be sure to register soon! We’ll look forward to seeing you in Baltimore, Maryland, March  3 – 6 at this terrific learning and networking event.



Charles Jennings on Measuring Informal and Workplace Learning: Questionmark 2013 Users Conference Keynote

Posted by Joan Phaup

We are delighted to announce that  Charles Jennings, one of the world’s leading thinkers and practitioners in learning and development, will deliver the keynote address at the Questionmark 2013 Users Conference, which will take place in Baltimore, Maryland, March 3 – 6.

His presentation,  “Meeting the Challenge of Measuring Informal and Workplace Learning,” will focus on the widespread adoption of the 70:20:10 framework —  based on studies that show  high performers learn approximately 70% from experience, 20% from others and 10% from formal study.

Charles Jennings

Charles will show how the 70:20:10 framework serves as a strategy for extending development beyond formal, structured learning to include informal and experiential learning opportunities. He will pose some important questions about which approaches support informal and workplace learning most effectively and how to effectively measure the success of those approaches.

As head of Duntroon Associates, Charles helps clients with learning and performance strategy, change management and with implementing improved approaches to workforce and leadership development. Previously, he served as Chief Learning Officer for Reuters and Thomson Reuters, where he led a team of 350 learning professionals for the firm’s workforce of 55,000.

This will be our 11th annual North American conference and, incidentally, a celebration of Questionmark’s 25th anniversary! We are busy planning the conference program and are welcoming case study and peer discussion proposals from experienced Questionmark users.

If you have a success story to tell or would like to get together with your colleagues to discuss a topic that concerns you, please check out our call for proposals and send in your ideas by September 14. Even if you are not yet sure you’ll be at the conference, that’s okay. We’d still like to hear from you!