In a previous post (NCCA Accreditations), you learned about the ARRT's accreditation through the National Commission for Certifying Agencies (NCCA). More recently, you learned about the Job Analysis process (NCCA Standard 14: Job Analysis), Examination Specifications (NCCA Standard 15: Examination Specifications), and Examination Development (NCCA Standard 16: Examination Development). Today, we continue our series exploring another standard that certification organizations must meet to earn accreditation.
Now that the Job Analysis is complete, the specifications around the examination have been identified, and examination has been developed, we can turn our attention to setting and maintaining the passing standard. Standard 17 specifically addresses how we do this, and states, "A certification program must establish a passing standard that relates performance on the examination to the level of proficiency required for certification. In addition, the program must use psychometrically defensible methods to help ensure that candidates are held to the same performance standard." The standard contains four essential elements, and ARRT takes specific steps to ensure we meet each one.
First, "The procedures that the program uses to establish performance standards must be based on generally accepted psychometric principles consistent with the purpose of the examination and item format(s) used."
Second, "The certification program must document the standard-setting process in sufficient detail to allow for replication. The documentation must include descriptions of the procedures and results. If a program considers this documentation confidential, they must make a general description of the methods they used in setting standards publicly available." ARRT Standard Setting Reports are publicly available on the ARRT website.
Third, "The program must evaluate standards of proficiency frequently enough to reflect current practice."
Forth, "The program must use statistical equating or other psychometrically sound procedures to hold candidates to the same performance standard across forms."
ARRT derives the Examination Content Specifications and Task Inventories directly from our practice analysis process and then uses those documents with our committees of subject matter experts (SMEs) to develop items (the questions on the exam) and then pulls items together in a systematic approach to create examination forms (the actual test). ARRT conducts the Standard Setting process approximately every five years in conjunction with the Job Analysis. Once we launch a new form based on the new content specifications resulting from a Job Analysis, we conduct the Standard Setting process to set the new pass point (i.e., we statistically determine how many items a candidate needs to get correct to pass the exam).
