About the role
About The Role Join Rex.zone’s AI Data Operations team to deliver high-quality labeled datasets that improve large language model evaluation and downstream model performance. You will label and review text, images, audio, and multimodal data while following strict annotation guidelines and quality targets.
Key Responsibilities
- Create accurate training labels for NLP and computer vision annotation tasks (text, image, audio, multimodal).
- Execute RLHF workflows (preference ranking, rationale writing, policy-aligned scoring) for LLM training pipelines.
- Perform QA evaluation, audit samples, and correct labeling errors to improve training data quality.
- Apply annotation guidelines compliance; document edge cases and reduce reviewer variance.
- Run prompt evaluation to score factuality, helpfulness, safety, and instruction adherence.
- Use taxonomies for content safety labeling (hate, harassment, self-harm, sexual content, violence) with consistent decision rules.
- Collaborate with project leads on rubrics, calibration sets, and acceptance criteria tied to model performance improvement.
Required Qualifications
- Mid-Senior experience in data labeling, data annotation, QA evaluation, or LLM evaluation.
- Strong written English for rubric-based judgments and rationale quality.
- Ability to follow annotation guidelines compliance and maintain consistency.
- Familiarity with NLP tasks such as named entity recognition and classification.
- Comfort working remotely in metrics-driven workflows.
Preferred Qualifications
- Exposure to RLHF and large language model evaluation methods.
- Experience with computer vision annotation (bounding boxes, polygons, segmentation).
- Experience labeling sensitive content with content safety policies.
Compensation
- Hourly rate: $30–$50 per hour
- Remote role across Canada
How To Apply Apply through Rex.zone and include relevant experience in data labeling, QA evaluation, and LLM/prompt evaluation.
Share examples of guideline-driven work (rubrics, calibration, audits) if available.
Millions of jobs, with real people getting hired every day
Questions, answered
Click "Apply with JobAssist" – we tailor your resume and application to this role and submit it for your approval.
Yes. This role at Rex.zone was screened before publishing – we confirmed the employer before listing it.
The employer didn't disclose a salary range for this listing. JobAssist shows pay whenever it's available.
This position can be done from anywhere, with no in-office requirement.
Yes – every application is tailored from your profile and this job's requirements, and you can review and edit before it's sent.