New Item Analysis Report: The Detail Page

jim_small Posted by Jim Farrell

In a recent post I explained the Summary page of the new Item Analysis report in Questionmark Analytics, now available in Questionmark OnDemand. Today I want to show you the question detail page.

ItemAnalysisReport2

This is the top of the detail page for a question. It first provides the high-level summary row from the previous page and then shows a number of statements that interpret the psychometric performance of the question based on the statistics. This gives users a high-level text description of what is going right or wrong with the question. This makes it easier for non-expert users to get actionable item analysis information, and makes it easy to identify the questions that you need to review.

They can then drill deeper as they scroll down the page for more statistical details regarding the question performance:

ItemAnalysisReport3

The p-values and item-total correlations have 95% confidence interval information in brackets next to the observed values. This gives users a sense of the error range around the statistics. (See  a previous blog article on using confidence intervals.)

There are additional stats as well, such as the Item reliability index that some psychometricians use. Below that is the answer option table showing for MC questions the numbers and percentages of participants that selected each response option. Below/beside that is a question preview for the question so that users can see what the question looked like inline with the statistics.

Remember, if you are running a medium or high stakes assessment that has to be legally defensible, then you cannot confirm that the assessment is valid if you are not running item analysis. And for all quizzes, tests and exams, running an item analysis report will give you information to help you make the assessment better.

Leave a Reply