You may have noticed the ARRT has posted new job opportunities in Education Requirements and Psychometrics (ERaP). Especially interesting to me is the psychometrician opening. Few people know what a psychometrician does and even fewer know how one fits into credentialing. So, let me tell you.
A credentialing psychometrician must first adopt more standards in their work. Usually, psychometricians are guided by The Standards for Educational and Psychological Testing, but, in credentialling, we are also guided by the National Commission for Certifying Agencies (NCCA) Standards for the Accreditation of Certification Programs. Only after studying the text is one ready to practice psychometrics in credentialing.
A psychometrician spends their time over the duration of one discipline's examination development building pools or examinations for committees to review, then building final examinations and troubleshooting them for enemies and cluing with the exam development coordinator (EDC). Before providing the scaling of the exam-that is, converting the number of correct responses into a scaled score that the candidate will see-each test question must first be calibrated, or set on the same underlying scale as all the other test questions in the discipline's bank. All these activities involve the use of statistical software, which is now an open-source language called R for most psychometric organizations. Other organizations may use off-the-shelf software or custom-built software. ERaP uses all three types of software to do our psychometric work.
Psychometricians can get buried in the theoretical details of the above work. (After all, there is a reason that there is a high educational burden placed on psychometricians.) But what is ultimately most important is providing a fair test with repeatable and meaningful results. At the ARRT, one psychometrician performs weekly processing and quality control checks to confirm that all scores given to candidates are correct. Finally, we document everything. Why did we revise a test question? Look in the documentation. Why did we change a test question's weight toward the scaled score? Look in the documentation.
The job is bigger still. The ARRT psychometricians perform and assist with practice analyses, standard setting, exam review committees, and day-to-day programming and analysis tasks that involve test data or metadata.
If this sounds like fun, and you happen to be-or know-a psychometrician, please let them know we're looking to expand our department. Here's the link.
