Psychometrician, Office of Assessment and Evaluation

About This Role

This psychometrician role is within the Office of Assessment and Evaluation (OAE) at the University of Arizona College of Medicine – Phoenix. The successful candidate will lead psychometric activities for examinations and assessments, while advancing methods and best practices in measuring medical knowledge, clinical skills, and overall MD program evaluation. This position also involves fulfilling the responsibilities of a faculty member in the Department of Biomedical Informatics, including active participation in teaching, clinical service, and scholarly work.

Responsibilities

• Perform item analysis, item response theory (IRT) calibrations, scoring, equating, standard setting, and validity and reliability studies across multiple assessments, including knowledge-based and clinical assessments in both pre-clerkship and clerkship phases of the MD program
• Conduct comprehensive assessment data analyses to support continuous quality improvement and ensure compliance with accreditation standards, including those of the Liaison Committee on Medical Education (LCME)
• Develop, maintain, and continuously improve operational psychometric processes to enhance efficiency, accuracy, and scalability
• Collaborate with subject matter experts and exam developers in item writing, item review, and the design, development, and evaluation of assessments
• Design and build robust infrastructure for assessment data architecture, enabling efficient querying, rapid data retrieval, and timely, accurate responses to data requests
• Collaborate on research and development initiatives within OAE and across the institution
• Disseminate findings to the broader medical education community through presentations and peer-reviewed publications
• Provide expert consultation on research design, measurement, statistical methodologies, and data systems
• Develop and maintain a comprehensive dashboarding system for student assessment data to support real-time insights and data-informed decision-making
• Other duties as assigned

Requirements

• PhD in educational or psychological measurements, statistics, or a related field from an accredited institution
• Minimum of one to three years of relevant experience
• Qualify for an appointment at the rank of assistant or associate professor
• Meet the requirements of a career-track faculty according to College and University guidelines

Nice to have:

• Proficiency in statistical analysis and IRT software, including SAS and R
• Strong knowledge and applied experience in psychometric principles, including item response theory (e.g., item calibration, item evaluation, linking/equating, and IRT-based scoring), classical test theory, and standard setting methodologies