Conceptual Assessment Framework: Building the Student Model
Posted by Austin Fossey
The student model is one of the three sections of the Conceptual Assessment Framework (CAF) in Evidence-Centered Design (ECD). In the student model, we define the variables from which we make inferences about the participant’s knowledge, skills, or abilities. We also define how those variables and inferences are related to each other.
The most basic form of a student model is a pass/fail indicator variable. The participant takes a test, and the test yields a pass/fail decision. These student model variables can be interpreted with respect to the proficiencies defined in a domain model for the assessment. For example, if the student passes, then we may infer that the student is proficient with the knowledge defined in the assessment’s domain.
Quite often, we have stakeholders who want more than just a pass/fail decision. Participants may want to know how close they were to passing, instructors may want to know which areas are strengths or weaknesses for their students, and researchers may want to classify participants based on similarities in response patterns.
In these cases, we need to define a more detailed student model. For example, instead of reporting a pass/fail indicator, we might report a score (e.g., percentage correct, scale score) so that participants understand how their performance relates to some criterion. We may also provide scores or outcomes for topics and subtopics within the assessment so that participants and instructors can look for patterns of strengths and weaknesses.
Student model diagram illustrating relationships between student model variables.
Evidence-Centered Assessment Design
(Mislevy, Steinberg, & Almond, 2012)
The relationships between the variables in the student model are important because they determine many of the inferences we will make about the participants. Topic and subtopic scores and outcomes are a good example of dependencies between student model variables: the overall assessment outcome and its inference may be dependent on the inferences about the topic scores.
For example, you may have an assessment where participants must demonstrate a level of proficiency in one topic area, otherwise they do not pass the assessment. Driving tests are an example of this. We make different inferences about a participant based on their performance on the written test and their performance on the road test, and the overall evaluation (i.e., awarding of a driver’s license) is dependent on both of these two student model variables.
Simplified example of student model diagram for a driver’s license test.
If you are interested in learning more about this, I’ll be at the Questionmark Users Conference in San Antonio March 4 – 7 and will be happy to talk with you.