[Remote] AI Trainer Jobs in Canada

Note: The job is a remote job and is open to candidates in USA. Rex.zone is seeking AI Trainers to improve large language models through various evaluation methods and data labeling. The role involves performing RLHF evaluations, executing prompt evaluations, and ensuring the quality of training data.

Responsibilities

• Perform RLHF evaluations (pairwise ranking, rubric-based scoring) and write clear rationales
• Execute prompt evaluation for instruction-following, factuality, and safety
• Label and validate datasets for NLP and content safety labeling
• Run QA evaluation checks (consistency, agreement, systematic error discovery)
• Document edge cases and build error taxonomies to drive model performance improvement
• Collaborate on rubrics, gold sets, calibration, and regression testing for model updates

Skills

• Experience with structured evaluation and guideline-driven judgment
• Strong writing and documentation for rationales and edge-case notes
• Familiarity with RLHF, LLM evaluation, and prompt evaluation workflows
• Comfort with data labeling, QA evaluation, and training data quality processes
• Bonus: multilingual evaluation, NER/classification tasks, or multimodal evaluation

Company Overview