About This Role
This role is for a high-taste engineer with a strong eye for quality, who has hands-on experience with preference data and rubric creation. The engineer will accelerate model development through high-quality training data, evaluations, and engineering talent, working with frontier AI labs.
Responsibilities
- Design, refine, and iterate on rubrics and evaluation criteria for preference data
- Review and label data with a high bar for quality, catching subtle issues others might miss
- Build and maintain pipelines/infra that support data generation, collection, and evaluation workflows
- Synthesize findings from data work into clear write-ups, updates, or recommendations for the team
- Collaborate closely with researchers/engineers to translate qualitative judgment into scalable processes
Requirements
- Proficiency in Python (especially building/maintaining pipelines and infra) or Typescript (experience contributing to or navigating large codebases like VSCode a plus)
- Hands-on experience with preference data collection and/or rubric creation
- Strong written and verbal communication — able to distill technical work into clear, actionable summaries
- Demonstrated high-taste judgment — able to articulate why something is good, not just that it is
Benefits
- Opportunity to work on cutting-edge AI projects with leading LLM companies
Important Details
- Availability: 40 hours per week preferred, with a minimum 6-hour overlap with Pacific Time (if unable to commit to 40 hours, share available hours and overlap window)
- Employment type: Independent contractor (does not include medical or paid leave benefits)
- Duration: 3 months
- Start date: as soon as possible
Application Process
- One round of interview (30 minutes of technical & cultural discussion)
About the Company
Turing is one of the world's leading AGI infrastructure companies, working with frontier AI labs.
