Item Analysis Report – High-Low Discrimination

Austin Fossey-42Posted by Austin Fossey

In our discussion about correlational item discrimination, I mentioned that there are several other ways to quantify discrimination. One of the simplest ways to calculate discrimination is the High-Low Discrimination index, which is included on the item detail views in Questionmark’s Item Analysis Report.

To calculate the High-Low Discrimination value, we simply subtract the percentage of low-scoring participants who got the item correct from the percentage of high-scoring participants who got the item correct. If 30% of our low-scoring participants answered correctly, and 80% of our high-scoring participants answered correctly, then the High-Low Discrimination is 0.80 – 0.30 = 0.50.

But what is the cut point between high and low scorers? In his article, “Selection of Upper and Lower Groups for the Validation of Test Items,” Kelley demonstrated that the High-Low Discrimination index may be more stable when we define the upper and lower groups as participants with the top 27% and bottom 27% of total scores, respectively. This is the same method that is used to define the upper and lower groups in Questionmark’s Item Analysis Report.

The interpretation of High-Low Discrimination is similar to the interpretation of correlational indices: positive values indicate good discrimination, values near zero indicate that there is little discrimination, and negative discrimination indicates that the item is easier for low-scoring participants.

In Measuring Educational Achievement, Ebel recommended the following cut points for interpreting High-Low Discrimination (D):

Capture Blog 18

In Introduction to Classical and Modern Test Theory, Crocker and Algina note that there are some drawbacks to the High-Low Discrimination index. First, it is more common to see items with the same p value having large discrepancies in their High-Low Discrimination values. Second, unlike correlation discrimination indices, High-Low Discrimination can only be calculated for dichotomous items. Finally, the High-Low Discrimination does not have a defined sampling distribution, which means that confidence intervals cannot be calculated, and practitioners cannot determine whether there are statistical differences in High-Low Discrimination values.

Nevertheless, High-Low Discrimination is easy to calculate and interpret, so it is still a very useful tool for item analysis, especially in small-scale assessment. The figure below shows an example of the High-Low Discrimination value on the item detail view of the Item Analysis Report.

high_low_discrimination

High-Low Discrimination value on the item detail page of Questionmark’s Item Analysis Report.

Integrating and Connectors – SuccessFactors

Doug Peterson HeadshotPosted By Doug Peterson

In this installment of Integrating and Connectors, I’d like to take a look at SuccessFactors. SuccessFactors is a very popular cloud-based human capital management application suite. It includes modules for succession planning, goals, performance management, recruiting, and – you guessed it – a learning management system (LMS) called Learning (appropriate, no?).

Questionmark assessments can be integrated into SuccessFactors Learning items by publishing them as AICC or SCORM content packages and importing the content package as a content object, which is then included in the learning item. The student logs into SuccessFactors Learning, enrolls in a course, takes in the content, clicks on a link and – voila! – the assessment launches. It’s a seamless experience for the student.

However, our integration with SuccessFactors Learning goes a step further. Learning and Questionmark Enterprise Manager can be connected by a Single Sign-On bridge that allows a Learning administrator to access Questionmark reports directly – no signing into Questionmark EM separately with some other ID and password.

This short video tells the story. Check it out and feel free to contact any of the Questionmark team if you have any questions.

Success Factors

Podcast: An Innovative Approach to Delivering Questionmark Assessments

 

Posted By Sarah Elkins

The University of Bradford has recently developed an innovative e-assessment facility, using cutting-edge thin client technology to provide a 100-seat room dedicated primarily to summative assessment. The room provides enhanced security features for online assessment and has been used for the first time in 2009 with considerable success. The room’s flexible design maximises its usage by allowing for formative testing, diagnostic testing and general teaching.

John Dermo is the e-Assessment Advisor at the University of Bradford.  In this podcast he explains the technology behind this unique setup and talks about the benefits and challenges in using this room. John Dermo will also be presenting a session at the 2009 European Users Conference, where he will go into more detail about the project.