Nine tips on recommended assessment practice — from Barcelona

John Kleeman HeadshotPosted by John Kleeman

Something I enjoy most about our users conferences is the chance to learn from experts about good practice in assessments. Most of our customers have deep knowledge and insightful practical experience, so there is always much to learn.

Here are some tips I picked up last week at our recent European Users Conference in Barcelona.Questionmark2013_DSC3209

1. Make sure to blueprint. It’s critical to have a detailed design (often called a blueprint) for an assessment – or as one user shared, “Without a blueprint, you don’t have an assessment”.

2. Network to get SMEs. With technology changing quickly, if your assessments assess IT or other new technology, the content changes very quickly and the quality of your subject matter experts (SMEs) who create and review items is critical. As an assessment owner, use networking skills to get the right SMEs on board; getting them engaged and building trust are essential.

3. Test above knowledge. Develop questions that test application or comprehension, for instance using scenarios. They are more likely to make your test valid than questions that simply test facts.

4. Give employees ownership of their own compliance testing. If employees have to take annual refresher tests, give them the responsibility to do so and encourage sel- learning and pre-reading. Give them plenty of time (e.g. 6 weeks’ warning), but make it their responsibility to take and pass the test in the window, not yours to keep on reminding them.

5. Gather feedback from participants. Make sure you solicit feedback from your participants on tests and the testing experience. That way you will learn about weak questions and how to improve your testing process. And you also make participants feel that the process is fairer.

6. Use job/task analysis. Asking questions about jobs and tasks is the best way to specify the criteria used to judge competency or proficiency. These questions can be automated in Questionmark right now. Watch this space for improvements coming to make this easier.

7. Look at Questionmark Live for item review workshops. If you have any informal or informal process for having groups of people working on or reviewing items, look at Questionmark Live. It’s free to use, has great group working capability and improves productivity. A lot of organizations are having success with it.

8. Keep feedback short and to the point… especially on mobile devices where people won’t read long messages.Questionmark2013_DSC3215

9. Look for live data, not just your rear view mirror. Data is important – without measurement we cannot improve. But make sure the data you are looking at is not dead data. Looking into the rear view mirror of what happened in the past doesn’t help as much as using reports and analytics from Questionmark to discover what is happening now, and use that data to improve things.

I hope some of these tips can help you in your work with assessments.

I will write in the spring about the tips I gather at the 2014 U.S. Users Conference in San Antonio!

Reflections on Barcelona: Great learning, great connections

Doug Peterson HeadshotPosted By Doug Peterson

I had the opportunity to attend and present at the Questionmark European Users Conference in Barcelona, Spain, November 10 – 12. Questionmark2013_DSC3268 If you’ve never had the chance to go to Barcelona, I highly recommend it! This is a *beautiful* city full of charming people, wonderful architecture, and GREAT food.

Traveling and seeing new sights and engaging in new adventures is always fun, but the true “stars of the show” at a users conference are, of course, the users.

It was wonderful to catch up with customers I first met two years ago in Brussels. I also had the opportunity to meet in person several people I had met over the last couple of years only through emails and conference calls, and it was great to put a face with a name. And of course, it was wonderful to make brand new friends whom I hope to see again next year!

One of the things I enjoy the most about Questionmark Users Conferences is how customers learn from each other. This happens in a more structured way during the many sessions presented by members of our user community, but I enjoy it even more when it happens more informally.

During the opening reception Sunday night I had the opportunity to talk with several customers, to hear why they were attending – what they wanted to get out of the conference – and then introduce them to another customer or a Questionmark employee who could help them meet their goals for the conference. Breakfast and lunch conversations were always interesting, and even during Monday night’s fantastic dinner at El Torre Dels Lleones, the conversations between different users facing various challenges continued (except, of course, when we were all watching the flamenco dancers perform!). These conversations are simply invaluable, not only because customers help each other find answers to their challenges, but because I as an employee gain insights and a depth of understanding as to what our customers are doing and the problems they are facing in ways that an email or a phone call can’t communicate.

Questionmark2013_DSC3125Tuesday morning we tried something new during the General Session. Howard Eisenberg gave an informative presentation on item writing best practices, and at certain points he would pause so that each table could discuss the current topic amongst ourselves. Then he would take comments from some of the tables before moving forward with the next topic. The conversations at the table where I was sitting were GREAT! The sharing of different perspectives and experiences resulted in a lot , “Oh, I never thought of that!” expressions all around the room. THAT’S why I love going to our users conferences:  there’s just nothing like the information exchange and growth that takes place when a bunch of Questionmark users gather together in one place.

If you weren’t able to make it to Barcelona, I hope you can come to San Antonio, Texas, March 4 – 7. We’ll be right on the city’s River Walk. I’ve been there several times visiting family in the area and, I can tell you it’s beautiful and FUN! Please join us. I’m confident that by the end of the conference you will agree that it was time well spent.

Integration Highlights in Barcelona

Steve Lay HeadshotPosted by Steve Lay

The programme for Questionmark’s European Users Conference in Barcelona November 10 – 12 is just being finalized. As usual, there is plenty to interest customers who are integrating with our Open Assessment Platform.

This year’s conference includes a case study from Wageningen University on using QMWISe, our SOAP-based API, to create a dashboard designed to help you manage your eu confassessment process. Also, our Director of Solution Services, Howard Eisenberg, will be leading a session oncustomising the participant interface so you can learn how to integrate your own CSS into your Questionmark assessments.

I’ll be running a session introducing you to the main integration points and connectors with the assistance of two colleagues this year: Doug Peterson will be there to help translate some of the technical jargon into plain English and Bart Hendrickx will bring some valuable experience from real-world applications to the session. As always, we’ll be available throughout the conference to answer questions if you can’t make the session itself.

Finally, participants will also get the chance to meet Austin Fossey, our Analytics Product Owner, who will be talking, amongst other things, about our OData API for Analytics. This API allows you to create bespoke reports from data ‘feeds’ published from the results warehouse.

See the complete conference schedule here, and sign up soon if you have not done so already.

See you in Barcelona!

Barcelona or San Antonio or both?

John Kleeman HeadshotPosted by John Kleeman

Questionmark users conferences are unforgettable. I’ve been to all 14 of them so far and each is engraved in my memory as an empowering, enriching and mesmerising event.  We are running two user conferences in the next few months and if you have a chance to attend one (or both!) I promise you won’t regret it.barcelona

Our first upcoming conference is the Questionmark European conference in Barcelona, Spain on 10-12 November. Barcelona is one of the most exciting cities in Europe and will be a great place to learn from other assessment professionals. You can see details at www.questionmark.com/uk/conference.River Walk San Antonio

Our second upcoming conference is the Questionmark US User Conference in San Antonio, Texas on 4-7 March, 2014. San Antonio is the home of the Alamo and the conference venue is part of the River Walk — a uniquely peaceful and positive environment for a conference. You can see details at www.questionmark.com/us/conference.

Here are five reasons I think Questionmark conferences are worth coming to:

1. Learn about assessments. I’ve been working with assessments for over 25 years … I know a lot, but  I’m still learning. Quizzes, surveys, tests and exams are hugely powerful ways of measuring human behavior and helping organizations improve. There is so much to learn.

2. Learn from Questionmark.  Our best presenters and technical experts are at the conference, and they have a lot to share.

Conversation at a Questionmark user conference3. Learn from peers. Most attendees say that the best thing about a Questionmark user conference is that they meet and learn from peers who have similar issues to them. A problem shared is often a problem solved, and you can find out what other people have done in their organization to solve the problems you are facing in yours.

4. Influence the future of the product. What we learn at these conferences contributes hugely to how we improve our products and services. Our product owners (people like Jim Farrell, Austin Fossey, Doug Peterson and Steve Lay)  attend the conferences and listen carefully to what our customers say.

5. Great cities. We know that people who come to our conferences go back to their organization passionate about online assessments and enthusiastic about wider use of Questionmark. We choose great venues for our conferences, and provide memorable experiences in a great environment so the conferences are fantastic personal experiences as well as being fulfilling learning opportunities.

I look forward to meeting readers of this blog at the conferences. And if any conference attendee can name the cities where the 14 conferences prior to these ones were held, I will buy you to a drink of your choice!

Get details here for the European Conference and here for the US Conference.

European Users Conference agenda and early-bird savings

Chloe MendoncaPosted by Chloe Mendonca

If you haven’t registered for the Questionmark 2013 European Users Conference, make sure you do so this month. Early-bird registration ends on September 30.  All it takes is a few clicks to set you on your way to some fantastic learning sessions, one-to-one meetings and networking in Barcelona November 10-12.

The program will cover an array of topics. Here are some of the breakout sessions we’ve scheduled to date:

g1Best Practice

  • ​Seven Reasons to Use Online Assessments for Regulatory Compliance
  • Assessment Feedback – What Can We Learn from Psychology Research?
  • Reporting and Analytics: Understanding Assessment Results

      

g4

Case Studies

  • Advanced Item Banking and Psychometric Analyses with Randomly Generated Assessments – ECABO
  • Get your Questionmark environment back in control with a QMWise based dashboard – Wageningen University
  • Questionmark Data Analysis for High Stakes Assessments – HBO Raad
  • Expanding and Testing Knowledge with Computer Assisted Learning – CFL (National Railway Company of Luxembourg)
  • Collaboration Facilities for Question Development Using QMlive​ – As-a-queue


g2Features and Functions

  • Introduction to Questionmark for Beginners
  • Assessment Authoring Part One: Creating Items and Topics,
  • Assessment Authoring Part Two: Assessments and Qpacks
  • Reporting and Analytics: Understanding Assessment Results
  • Integrating with Questionmark’s Open Assessment Platform
  • Customising the Participant Interface ​ ​
  • Deploying Questionmark Perception On Premise


g3Drop in Discussions and Demos

  • Drop-in Demo Session: Latest Questionmark Features
  • Tips for Delivering your Assessments to Mobile Devices

More Sessions will be added soon!

Conference participation is a valuable investment. Visit our cost justification page and download the ROI Toolkit to see why this is an essential experience for you and your organisation.

Barcelona banner

Grow what you know about assessment and measurement

Joan Phaup HeadshotPosted by Joan Phaup

We have not just one but two important opportunities coming up for customers to find out how to use our technologies more effectively, discuss best practices in assessment and measurement and soar up the learning curve.

The  Questionmark 2013 European Users Conference  in  Barcelona November 10 – 12 and the Questionmark 2014 Users Conference, in San Antonio March 4 – 7 both offer tremendous professional development opportunities, along with the chance to influence the future of our technologies.

We’ll tell you time and again that these conferences are the best places to learn, network and discover new tips and techniques — but don’t take our word for it.

Find out from this video what participants thought about our most recent conference — and start making your plans to attend!

[youtube http://www.youtube.com/watch?v=kcSN7qUa0zc?list=UUG2RlPbdbq1AO051Ityq9Wg]