Getting the results you want by creating your own report filters

Posted by Kate Soper

When reporting on assessment outcomes, it’s often desirable to compare the performance of one  group  to that of another.  Perhaps you’d like  to filter the results of an evaluation according to the demographics of the participants or according to the course instructor, location or participant group.

Questionmark Perception makes this possible by enabling you define your own filters. For instance, you can identify certain questions in an assessment as demographic questions and have the answers to those questions saved in Special Fields that are set aside specifically for filtering data from the results. This means that questions asking participants to identify variables like their trainers or departments can then be used to filter your report results; the results you see only match participants who selected those answers.

You may designate up to 10 Special (demographic) Fields per assessment:

In addition to your demographic fields, you can apply filters for dates, date ranges, participant names, topics, and assessments so that you can view the information you need when you run your reports.

By the way, when creating a course evaluation in Questionmark Live, you can add questions designated as “demographic questions.” By default, any demographic question will automatically save to a special field.  Once you import the assessment into Questionmark Perception, you can change which special field that question data is saved to.

The use of filters is just one aspect of reporting in Questionmark Perception. For general information about Perception reports, click here.  Questionmark Support Plan customers can get details about using Special Fields with Perception in this Knowledge Base article.

Conference Close-up: Perfecting the Test Through Question Analysis

Posted by Joan Phaup

Neelov Kar

Neelov Kar

Neelov Kar, Project Management Program Owner for Dell Services (previously Perot Systems) is getting ready to attend the Questionmark Users Conference in Miami this month. He will be delivering a case study about how he and his team have used statistical analysis to improve their test questions. I spent some time talking with Neelov the other day and wanted to share what I learned from him.

Q: Tell me a little about your company.

A:  We are a one-stop shop for IT Services and have people working all over the world, in 183 countries.

Q: What does your work entail?

A: I’m the project management program owner, so I am in charge of all the project management courses we offer. I help identify which courses are appropriate for people to take, based on training need analysis, and I work with our project management steering committee  to work out what courses we need to develop. Then we prioritize the requirements, design and develop the courses, pilot them and finally implement them as a regular course. As a Learning and Development department we also look after leadership courses and go through a similar process for those. I moved into this role about a year ago. Prior to that I was leading the evaluation team, and it was during my time on that team that we began using Questionmark.

Q: How you do you use online assessments?

A: We use Questionmark Perception for Level 2 assessment of our project management and leadership courses. We started with a hosted version of Questionmark Perception and it was I who actually internalized the tool. We offer leadership courses and project management courses internally within the organization across all geographies. Some of the project management courses already had tests, so we converted those to Questionmark.  We started designing the end-of-course assessments for our newly introduced leadership and project management courses once we started using Perception.

Q: What you will be talking about during your conference presentation?

A:  Last year we introduced a new course named P3MM Fundamentals, and because it was a new course we had to pilot the course with some of our senior members. In the pilot we asked the students to take the end-of-course test, and we found that many people had trouble passing the test. So we analyzed the results and refined the questions based on the responses. Analysis of results within Perception — particularly the Assessment Overview Report, Question Statistics Report, Test Analysis Report and Item Analysis Report — helped us in identifying the bad questions. We also saw that there were things we could do to improve the instruction within the course in order to better prepare people for the test. Using the Questionmark reports, we really perfected the test. This course has been going for over a year now, and it’s pretty stable. Now, every time we launch a course we do a pilot, administer the test and then use the Questionmark tools to analyze the questions to find out if we are doing justice to the people who are taking the test.

Q: What are you looking forward to at the conference?

A: I want to find out what Perception version 5 offers and how we can use it for our benefit.  Also, I saw that there are quite a few good papers to be presented, so I’m looking forward to attending those. And I want to get involved in the discussion about the future of SCORM.

Neelov’s is just one of 11 case studies to be presented at the conference, which will also include technical training, best practice presentations, peer discussions and more. Online registration for the conference ends on Tuesday, March 9th, so if you would like to attend, be sure to sign up soon!