Dance Video, Deadlines and Agenda – oh my!

The 2016 Questionmark Conference will be a seriously fun and jam packed learning event in Miami April

sunWhat’s on the agenda? Here’s a sneak-peek of what you can expect:

  • Secure Authentication: Accessing Questionmark OnDemand with SSO
  • Role-based Security in Questionmark OnDemand
  • Taking Your Test Planning to the Next Level: JTAs and Blueprints 
  • Capturing Demographics When Delivering Assessments

sunAre you a Questionmark superstar? Share your story in Miami!

  • Do you have an experience you would like to share, or a topic to discuss? Submit your proposal by Friday (December 4) and become a speaker at the Conference. *Slots are limited
  • Check out this dance video and get in the mood to learn and network alongside your peers in sunny Miami!

Invite Video

Coming soon to South Africa: A full conference programme by and for Questionmark users!

Chloe MendoncaPosted by Chloe Mendonca

Questionmark users gathering in Midrand, South Africa 21-22 August have a variety of sessions to look forward to. The programme for the South African Questionmark Users Conference is almost finalised and now includes a range of customer case studies as well as sessions about Questionmark features and functions.

We’re especially pleased to have Jim Farrell, who leads Questionmark’s product management
team, on board as a presenter. Jim has more than 20 years’ experience in the world of education and assessments and will share some of the latest
developments in Questionmark’s browser-based authoring tool and cover the newest Reporting and Analytics features.

This event is for Questionmark customers as well as individuals interested in learning more about Questionmark’s technologies.

Why attend? Here are some excellent reasons:

  • Write better assessments: Questionmark experts will share advice about effective assessment authoring
  • Learn from your peers through case studies: discover strategies that have helped other users to save time and money and compare your organisation’s practices to theirs
  • Influence the product roadmap: explain to Questionmark’s product managers what you need from your assessment software
  • Get face time: there is no substitute for face-to-face meetings with people who understand the opportunities and challenges in your work
  • Professional development opportunity: take advantage of this opportunity close to home

If you’ve already made plans to be there, we’re looking forward to seeing you! For those who haven’t signed up, we hope you will register and join us!


Nine tips on recommended assessment practice — from Barcelona

John Kleeman HeadshotPosted by John Kleeman

Something I enjoy most about our users conferences is the chance to learn from experts about good practice in assessments. Most of our customers have deep knowledge and insightful practical experience, so there is always much to learn.

Here are some tips I picked up last week at our recent European Users Conference in Barcelona.Questionmark2013_DSC3209

1. Make sure to blueprint. It’s critical to have a detailed design (often called a blueprint) for an assessment – or as one user shared, “Without a blueprint, you don’t have an assessment”.

2. Network to get SMEs. With technology changing quickly, if your assessments assess IT or other new technology, the content changes very quickly and the quality of your subject matter experts (SMEs) who create and review items is critical. As an assessment owner, use networking skills to get the right SMEs on board; getting them engaged and building trust are essential.

3. Test above knowledge. Develop questions that test application or comprehension, for instance using scenarios. They are more likely to make your test valid than questions that simply test facts.

4. Give employees ownership of their own compliance testing. If employees have to take annual refresher tests, give them the responsibility to do so and encourage sel- learning and pre-reading. Give them plenty of time (e.g. 6 weeks’ warning), but make it their responsibility to take and pass the test in the window, not yours to keep on reminding them.

5. Gather feedback from participants. Make sure you solicit feedback from your participants on tests and the testing experience. That way you will learn about weak questions and how to improve your testing process. And you also make participants feel that the process is fairer.

6. Use job/task analysis. Asking questions about jobs and tasks is the best way to specify the criteria used to judge competency or proficiency. These questions can be automated in Questionmark right now. Watch this space for improvements coming to make this easier.

7. Look at Questionmark Live for item review workshops. If you have any informal or informal process for having groups of people working on or reviewing items, look at Questionmark Live. It’s free to use, has great group working capability and improves productivity. A lot of organizations are having success with it.

8. Keep feedback short and to the point… especially on mobile devices where people won’t read long messages.Questionmark2013_DSC3215

9. Look for live data, not just your rear view mirror. Data is important – without measurement we cannot improve. But make sure the data you are looking at is not dead data. Looking into the rear view mirror of what happened in the past doesn’t help as much as using reports and analytics from Questionmark to discover what is happening now, and use that data to improve things.

I hope some of these tips can help you in your work with assessments.

I will write in the spring about the tips I gather at the 2014 U.S. Users Conference in San Antonio!

Barcelona or San Antonio or both?

John Kleeman HeadshotPosted by John Kleeman

Questionmark users conferences are unforgettable. I’ve been to all 14 of them so far and each is engraved in my memory as an empowering, enriching and mesmerising event.  We are running two user conferences in the next few months and if you have a chance to attend one (or both!) I promise you won’t regret

Our first upcoming conference is the Questionmark European conference in Barcelona, Spain on 10-12 November. Barcelona is one of the most exciting cities in Europe and will be a great place to learn from other assessment professionals. You can see details at Walk San Antonio

Our second upcoming conference is the Questionmark US User Conference in San Antonio, Texas on 4-7 March, 2014. San Antonio is the home of the Alamo and the conference venue is part of the River Walk — a uniquely peaceful and positive environment for a conference. You can see details at

Here are five reasons I think Questionmark conferences are worth coming to:

1. Learn about assessments. I’ve been working with assessments for over 25 years … I know a lot, but  I’m still learning. Quizzes, surveys, tests and exams are hugely powerful ways of measuring human behavior and helping organizations improve. There is so much to learn.

2. Learn from Questionmark.  Our best presenters and technical experts are at the conference, and they have a lot to share.

Conversation at a Questionmark user conference3. Learn from peers. Most attendees say that the best thing about a Questionmark user conference is that they meet and learn from peers who have similar issues to them. A problem shared is often a problem solved, and you can find out what other people have done in their organization to solve the problems you are facing in yours.

4. Influence the future of the product. What we learn at these conferences contributes hugely to how we improve our products and services. Our product owners (people like Jim Farrell, Austin Fossey, Doug Peterson and Steve Lay)  attend the conferences and listen carefully to what our customers say.

5. Great cities. We know that people who come to our conferences go back to their organization passionate about online assessments and enthusiastic about wider use of Questionmark. We choose great venues for our conferences, and provide memorable experiences in a great environment so the conferences are fantastic personal experiences as well as being fulfilling learning opportunities.

I look forward to meeting readers of this blog at the conferences. And if any conference attendee can name the cities where the 14 conferences prior to these ones were held, I will buy you to a drink of your choice!

Get details here for the European Conference and here for the US Conference.

Webinar: 7 good reasons to use online assessments for compliance

Joan Phaup HeadshotPosted by Joan Phaup

Many of our customers use online assessments to measure employees’ knowledge and understanding of regulations.

Such assessments can help promote ethics, compliance and risk mitigation, and can provide important information about culture and ethics across an entire organization. Tests, quizzes and surveys can easily reach every employee and gather information from them; they are powerful tools in the quest for compliance.

We are pleased to announce a webinar later this month on  7 Reasons to Use Online Assessments for Compliance and hope you will be able to join us for it on Wednesday, September 18th, at 11 a.m. EDT  (4 p.m. BST).

Questionmark Chairman John Kleeman and Marketing Director Brian McNamara will lead this one-hour webinar, which will explore the rationale for using online assessments in compliance, note some  examples of how they are being used effectively and offer pointers for incorporating them within a compliance program.

The presentation will also address these topics:

  • The compliance, risk management and business impact of online assessments
  • Presenting scenario-based questions that measure how employees respond to real-life situations
  • Providing feedback at the topic level to show employees where they need to improve

Go to our UK website  or our  US website for webinar details and free registration


Celebrating 25 years of change — from DOS to SaaS

Question Mark for DOS (1988)John Kleeman HeadshotPosted by John Kleeman

Considering all the security, availability and flexibility we can achieve today with cloud-based assessment management systems, it’s remarkable to look back at the many changes and milestones we’ve seen over the past 25 years.

I wrote the first version of Question Mark for DOS in 1987-88. When I launched the company, 25 years ago, in London in August 1988, I always wanted to bring the benefits of computerized assessment to the world, but it was hard to foresee the dramatic technological changes that would transform our industry and make online assessment as widespread as it is today.

Coinciding with the rise of the PC, Question Mark for DOS empowered trainers and teachers to create, deliver and report on computerized assessments without having to rely on IT specialists.

Question Mark Designer for Windows (1993)QM Web 1995Things have been changing quickly ever since.  The early 1990s brought the move from DOS — functional but boring — to Windows — visual and graphical. This was radical at the time. To quote our marketing for Question Mark Designer for Windows, launched in 1993:

“Using Question Mark Designer, you can create tests using the full graphical power of Windows. You can use fonts of any size and type, and you can include graphics up to 256 colours. One of the most exciting features is a new question type, called the “Hot spot” question. This lets the student answer by “pointing” at a place on the screen.”

The switch to a visual user interface was huge, but the biggest paradigm shift of all was the move to delivering assessments over the Internet.

Pre-Internet, communicating results from assessments at a distance meant sending floppy disks by post. The World Wide Web made it possible to put an assessment on a web server, have participants answer it online and get instantly viewable results. This changed the world of online assessments forever.

QuestionmQuestion Mark Perception (1998)ark Technical Director Paul Roberts, who still plays an important role in Questionmark product development, wrote the code for the world’s first-ever Internet assessment product, “QM Web”, in 1995.  We followed up QM Web with the first version of Questionmark Perception, our database-driven assessment system, in 1998.

Eric Shepherd founded the U.S. division of Questionmark in the 1990s and in 2000 became CEO of what is now a global company. He is the heart and soul of Questionmark, an inspiring chief executive who has turned Questionmark from a small company into an industry leader.

One key paradigm shift in the 2000s was the desire to use surveys, quizzes, tests and exams in more than one department — across the entire enterprise. To make this practical, we began building scalability, reliability, translatability, accessibility, maintainability and controllability into our technologies. These attributes, along with multiple delivery options and in-depth reporting tools, are key reasons people use Questionmark today.

Cutting the ribbon at the Questionmark European Data Center

Opening our European Data Centre last year marked a major expansion of our cloud-based Questionmark OnDemand service

In recent years, we’ve seen another dramatic change – towards software-as-a-service applications in the “cloud”.  Just as Question Mark for DOS 25 years ago empowered ordinary users to create assessments without needing much IT support, so Questionmark OnDemand today allows easy creation, delivery and reporting on assessments without in-house servers.

So what’s in store for the future? Technology is making rapid advances in responsive design, security, “big data”, mobile devices and more. Questionmark keeps spending around 25% of revenues on product development. The huge demand for online assessments is making this our busiest time ever, and we expect continued, rapid improvement.

I’d like to thank our customers, suppliers, partners, users and employees – whose support, collaboration and enthusiasm have been critical to Questionmark’s growth during our first 25 years. I look forward to continuing the journey and am eager to work with all of you to shape what happens next!