Online conference registration ends March 13: Here’s the program!

Posted by Joan Phaup

Online registration for the Questionmark 2012 Users Conference ends next Tuesday, March 13.

If you have not yet signed up, all it takes is a few mouse clicks and keystrokes to set you on your way to some great learning and networking in New Orleans March 20 – 23.

The festivities begin with two pre-conference workshops on March 20: a Boot Camp for people who want to learn the basics of using Questionmark technologies and a session on Criterion-Referenced Test Development by Sharon Shrock and Bill Coscarelli. There will also be three special interest group meetings that afternoon and a big reception that evening.

Then it’s on to a full conference program:

General Sessions:

Case Studies

  • An Architect’s Approach to Questionmark Assessment Development: How to Architect, Design and Implement an Efficient Assessment-Building Process – Tom Metzler, TIBCO Software, Inc.
  • How e-Testing is Improving Assessment for the U.S. Coast Guard – James Parry and Art Stark, USCG Training Center
  • Analysis of Examination Time Data at the Question- and Person-Specific Level with Perception – James J. Thompson, Louisiana State University
  • The Big Switch: Moving Training and Assessment to Mobile Devices – Barry Jass, Covidien
  • Using Questionmark to Answer Business Questions Related to Your Work Processes – Gretchen Seyfried, Progressive Insurance
  • Five Approaches to Effective Content Management – Sami Besalel, The Aurelius Group
  • Eighty’s a Crowd: Managing a Multitude of Assessment Authors – Scott Wilde, Intermountain Healthcare
  • Using Innovative Technologies to Aid High-Volume Testing in Multiple Environments – Jennifer Lathem and Kerry Eades, Oklahoma Department of Career and Technology Education

Features & Functions

  • Introduction to Questionmark Perception for Beginners
  • Answers to Windows Authoring FAQs
  • Using Questionmark Live for Surveys and Course Evaluations
  • Reporting and Analytics: Understanding and sharing assessment results
  • Deploying Questionmark Perception 5.4 OnPremise
  • Questionmark Live Bring-your-own-laptop Tutorial
  • Customizing the Participant Interface

Best Practices

  • Alignment, Impact and Measurement with the A-model – Dr. Bruce C. Aaron, Ametrico
  • Instructional Design for the Real World – Dr. Jane Bozarth, State of North Carolina
  • Applying the Principles of Item and Test Analysis to Your Assessment Program – Sean Farrell, PwC
  • Timing is Everything : Using psychology research to make your assessments more effective – John Kleeman, Questionmark
  • Using Captivate and Flash Simulations in eLearning and Assessments – Doug Peterson, Questionmark
  • Using Web Services to Integrate with Questionmark Perception – Steve Lay, Questionmark

Peer Discussions

  • Proctored versus non-proctored: How does assessment setting affect student achievement on web-based assessments? – Richard Pierce, Shenandoah University
  • Managing Assessment Security as the Stakes are Getting Higher – Questionmark CEO Eric Shepherd, Moderator
  • Test Defensibility in the US Coast Guard – Using the Angoff Method to Set Cut Scores – Alan H. Wheaton and and James R. Parry
  • Managing Multiple Assessment Authors – Scott Wilde, Intermountain Healthcare

Drop-in Demos

  • Delivering Assessments to Mobile Devices
  • Making the most of new features in Perception 5.3 and 5.4

Future Solutions

  • Focus Group: Authoring & Delivery
  • Focus Group: Open Assessment Platform and Analytics
  • Open Sessions

There’s still time to register! So do it now!

Conference Close-up: An enterprise architect’s approach to assessment development

Posted by Joan Phaup

I’m intrigued by the ingenuity of Questionmark users, who bring so many different perspectives to their work with learning and assessments.

Tom Metzler, for example, an enterprise architect and senior education consultant at TIBCO Software, Inc., brings many years’ experience in software design to his current work.  He is also the knowledge assessment administrator.

He’ll be sharing his insights at the Questionmark Users Conference in New Orleans March 20 – 23, during a case study presentation called An Architect’s Approach to Questionmark Assessment Development : How to Architect, Design and Implement an Efficient Assessment-Building Process.

Tom Metzler

His aim, Tom says, is to help fellow Questionmark users apply some principles of enterprise architecture to planning and building assessments. Here’s a quick account of a conversation I had with him last week.

How are you using assessments at TIBCO?

Our knowledge assessments are designed to verify a participant’s knowledge of key concepts and capabilities covered during TIBCO courses. This is a good method for showing participants and their companies concrete evidence of learning.

Our assessments are also very popular among our own employees who take them to help maintain their skill sets in this fast-changing technical environment.

Why do you think it’s important for assessment professionals to understand something about business architecture processes?

First, many business processes within many organizations are implemented ad hoc. Over time, these systems become brittle, inefficient, lack scalability and are generally unmanageable and undocumented. Knowledge Assessment processes are not immune. In essence, the business architecture is weak. It’s a house of cards. It is crisis management …putting out fires.

Second, since I first started writing code in the early ’70’s, automating business processes has evolved from a “just write the code” mentality to a mature industry. I believe computer science has had as much to do with this evolution as did learning from our mistakes.

By applying proven techniques, methodologies, and so forth to everyday business processes, people can save themselves a lot of time, improve the quality of their work, and ensure an attractive ROI. For example, they can use the conceptual principles of service-oriented architecture. These principles translate beautifully into improving processes for creating assessments, and you don’t have to be a computer scientist to do this kind of thing.

For instance?

One thing I’ll focus on during my conference presentation is the concept of discovering and building reusable components within the Questionmark Perception repository using proven software development techniques. I’ll also address the mistakes I initially made and how to avoid them.

What would be your top three bits of advice for someone who wants to apply these ideas?

1.    Obtain executive sponsorship
2.    Follow architecture best practices
3.    Document your processes

How can your own experience help your audience?

Because we are a software company, with a proven record, we understand what good software development means. I want to share this with the audience and give it to them in a form they can understand, so that they can capitalize on what we’ve done.  We architect our product solutions so it is natural to us to also architect our everyday business processes, which is what we did for our knowledge assessment program.

Who would benefit from hearing your presentation?

I think it would be helpful to people who are just starting out, since it would help them avoid some pitfalls and get started on the right foot. And it would be of particular interest to assessment development managers, repository managers, project managers and, business analysts and project architects.

What would you like people to take away from your session?

I hope they’ll see the value of architecting their Questionmark assessment-building process and get some ideas for implementing reusable components in their own assessments — a great way to ensure ROI. I really want them to be able to capitalize on the concepts I present.


Tom is one of several case study presenters at this conference. Check out the complete line-up of case studies, best practice presentations and other sessions hereand register soon!

Conference close-up: Assessment as an integral part of instructional design

Jane Bozarth

Posted by Joan Phaup

With the Questionmark Users Conference now less than a month away, it’s a good time to check out the conference agenda and — if you haven’t already done so — to sign up for three great days of learning and networking in New Orleans March 20 – 23.

Two high points on the program will be presentations by Dr. Jane Bozarth:

  • a keynote address on the importance of  starting with good objectives and clear outcomes for assessments and using them to strategically to support organizational goals
  • a breakout session called Instructional Design for the Real World —  about tools and tricks that support rapid instructional design, help  with needs analysis and make for effective communication with subject matter experts, managers and others

As a training practitioner for more than 20 years, and as Elearning Coordinator for the North Carolina Office of State Personnel, Jane will bring a lot of firsthand experience to these presentations. During a conversation I had with her shortly after she agreed to present at the conference, Jane pointed out some common pitfalls  that she will address during her keynote to help listeners address the right things at the right time for the right outcome:

  • getting so caught up in writing objectives and developing  instruction as to lost sight of the desired end result
  • measuring the wrong things or things that have insignificant impact
  • paying too little attention to formative assessment
  • waiting until after a product is designed to go back and write the assessment for it, instead of addressing assessment first

You can listen to the podcast of our conversation right here and or read the transcript.

Conference Close-up: Alignment, Impact & Measurement with the A-model

Posted by Joan Phaup

Key themes of the Questionmark Users Conference March 20 – 23 include the growing importance of informal and social learning — as reflected by the 70+20+10 model — and the role of assessment in performance improvement and talent management. It’s clear that new strategies for assessment and evaluations are needed within today’s complex workplaces.

Dr. Bruce C. Aaron

We’re delighted that measurement and evaluation specialist Dr. Bruce C. Aaron will be joining us at the conference to talk about the A-model framework he has developed for aligning assessment and evaluation with organizational goals, objectives and human performance issues.

A conversation Bruce and I had about A-model explores the changes that have taken place in recent years and today’s strong focus on performance improvement.

“We don’t speak so much about training or even training and development anymore,” Bruce explained. “We speak a lot more about performance improvement, or human performance, or learning and performance in the workplace. And those sorts of changes have had a great impact in how we do our business, how we design our solutions and how we go about assessing and evaluating them…We’re talking about formal learning, informal learning, social learning, classroom, blended delivery, everything from online learning to how people collect information from their networks and the knowledge management functions that we’re putting in place.”

In a complex world that requires complex performance solutions, Bruce observed that “the thing that doesn’t change is our focus on outcomes.”

The A-model evolved out of dealing with the need to stay focused on goals to logically organize the components of learning, evaluation and performance improvement. It’s a framework or map for holding the many elements of human performance in place — right from the original business problem or business issue up through program design and evaluation.

You can learn more about this from Bruce’s white paper, Alignment, Impact and Measurement with the A-model, from this recording of our conversation — and, of course, by attending the Users Conference! Register soon!

Conference Close-up: Using mobile devices for sales training and testing

Posted by Joan Phaup

The idea of using mobile devices for learning and assessment sparks many lively conversations these days – and we’re looking forward to discussing this subject during the Questionmark Users Conference March 20 – 23 in New Orleans.

Among those who can talk about this in detail is Barry Jass from Covidien Vascular Therapies, where Apple iPads and iPhones are used for sales training, product knowledge tests and observational assessments.

I enjoyed asking Barry about the case study he’ll be presenting at the conference.

Barry Jass

Barry Jass

Can you tell me about your job role?

I am the sales training manager in charge of elearning and curriculum development for our peripheral vascular division. I develop curriculum for the various products we have to keep our sales force up to the date on — new features, new indications, new product design and usage.

What do you think are the biggest challenges of training itinerant sales people?

I think the greatest challenge is that most people in sales don’t have much time. They are busy with their customers! For example, our sales force works directly with doctors; they’re serving as consultants for the use of our devices, so they don’t have much time for extended learning. They need the learning, but it’s hard for them to find the time to dedicate to it.

What will you cover during your presentation?

I’m going to talk about the concepts involved in training somebody through a mobile device. I think that’s a big issue for curriculum designers who’ve grown up thinking the computer was the greatest thing in training. Now we have tablets and smartphones!  Very different rules are necessary to make learning effective on a small device like a phone. It is more than simply taking already existing learning modules and making them available on a person’s phone.  I want to talk about how you plan learning and assessments that work on mobile devices. If there are people in the audience who are already using mobile assessments, I’ll want them to share their ideas throughout the session.

I’ll talk about the principles, but I’ll also give participants the chance to make an assessment or survey right there on the spot that can be used immediately. I’d like people to bring their phones and tablets with them so they can try this out using Questionmark OnDemand. I think that if you’re going to use the smartphone or tablet to view something, you should also be able to create it using the same tool.

Do you have any quick pointers for people pondering a move to mobile assessments?

My favorite quote is by Blaise Pascal: “The present letter is a very long one, simply because I had no leisure to make it shorter.” One of the hardest things to do is to make something short! We tend to create modules that are 45 minutes or an hour long, but for mobile learning these modules should be no more than five minutes. You also have to think about short assessments – say 3 questions instead of 10! That’s a big challenge, and it takes a lot more planning and development than a learning that would take place through your computer.

What do want people to take away from your session?

I hope they will learn that m-learning is here to stay. It’s not going back to the bigger screen: it’s going to a smaller screen. I think we have to embrace that and make those changes now or we’ll be left behind as instructional designers and curriculum writers.

I hope I can show participants how easy it is to use new technologies to create mobile learning and assessments.  I hope to give them some clear principles to work with as well as the chance to put them into action.

What are you most looking forward to about the conference?

Getting together with other people and hearing what they’re doing. I want to know the latest and greatest that’s out there because I always want to be on the cutting edge.


Want to get together with your colleagues and be on the cutting edge, too?  Register for the conference soon!

Early-bird alert: Last day for saving on conference registration

Posted by Joan Phaup

This is the last time I’ll mention early-bird sign-ups for the Questionmark 2012 Users Conference.

That’s because today is the last day to save $100 on your conference registration.

We are looking forward to seeing customers in New Orleans March 20 – 23 for three days of exceptional learning, professional development and networking.

A few highlights from the conference program:

Why wait any longer? Sign up today!