About This Role
This role is for a Psychometrician to design and conduct psychometric analyses for the WIDA ACCESS program, focusing on test performance, construction, validity, reliability, equating, and scoring. The position involves preparing technical reports, developing quality control procedures, and providing quantitative guidance to research staff and external resources. It suits a candidate with strong experience in psychometrics for high-stakes testing.
Responsibilities
- Design and conduct psychometric analyses for the WIDA ACCESS program
- Plan, conduct, document, and present psychometric and statistical analyses using classical test theory and the Rasch model with Winsteps and Facets software, including analyses related to item quality, rater performance, DIF, reliability and classification consistency, standard setting, equating, scoring, and reporting
- Investigate and evaluate new approaches to conducting psychometric analyses in language testing
- Establish and monitor quality control procedures related to quantitative analyses, from data cleaning to final technical reports, ensuring the reliability and integrity of results
- Provide guidance to professional researchers on proposal design and quantitative analytical approaches upon request
- Participate in writing quantitatively-oriented portions of proposals as needed
- Support outreach and collaboration, including research, development and client development activities with external partners, academia, innovators in education and language, and the public as needed
Requirements
- Ph.D. in Educational Measurement, Psychometrics, or related field
- Minimum of five years of demonstrated operational experience in designing and conducting statistical and psychometric analyses for high stakes testing programs
- Strong demonstrated experience in applying the Rasch model to a variety of psychometric issues
- Experience in conducting analysis on performance and polytomously-scored tasks
- Demonstrated proficiency in programming in R or Python
- Proficiency in using SPSS, Winsteps, Facets, and other statistical and psychometric software
- Ability to create and apply quality control procedures in creating and maintaining databases, conducting statistical and psychometric analyses, conducting other quantitative research, and producing technical reports
- Ability to program, document procedures, and instruct junior members of the team in statistical software such as R, Python, SPSS, Winsteps, Stata, and SAS
- Ability to create replicable procedures for routine psychometric work
- Ability to write narrative to accompany psychometric analyses for technical reports and other documentation
Nice to have:
- Experience working within a psychometric program in a K-12 U.S. Educational Context, and preferably in language testing