Standard setting is the process of establishing a minimum passing score on an examination. For ARRT exams, standard setting ensures that individuals who obtain an ARRT certification have demonstrated the knowledge and skills needed to practice effectively as a technologist. This process is essential for protecting patient safety, providing high-quality care, and upholding the integrity of the profession. By defining a clear threshold that separates qualified candidates from those who require further preparation, standard setting maintains a fair exam that reflects current practice in a discipline.
At its heart, standard setting combines expert judgment with statistical analysis to arrive at a cut score. Volunteer subject matter experts (SMEs) first identify the knowledge, skills, and abilities needed to be minimally qualified for certification. ARRT staff then lead SMEs through standardized exercises, such as the modified Angoff procedure. In this procedure, SMEs independently consider items on a recent form and decide, for each item, the percentage of minimally qualified candidates expected to answer the item correctly. For each item, staff average the percentages that SMEs selected, calculating an item average. Staff then average together all item averages to calculate a form's minimum passing percentage. Through an iterative process and further discussion, SMEs refine their judgments, and staff calculate an Angoff-derived cut score. While this cut score is a good starting point, ARRT takes a holistic approach and uses additional exercises to achieve the best cut score result. Specifically, ARRT staff guide SMEs through additional psychometric procedures and discuss the results together. Finally, staff elicit final cut score judgments from SMEs and identify a cut score to be recommended to the ARRT Board of Trustees, who choose a new standard for a discipline.
By pooling together expert judgments, standard setting ties real-world experience to a precise cut score that helps to define what it means to be competent. ARRT conducts a standard setting for each discipline periodically to ensure exams reflect changes in a discipline, especially as technologies evolve over time. The outcome is a fair exam, giving us confidence that passing candidates are qualified to practice as technologists.
