Senior Data Labeling Jobs in Ottawa (Remote)

Overview

Rex.zone is hiring for Senior Data Labeling roles supporting AI/ML training workflows end to end, including data labeling, RLHF, prompt evaluation, QA evaluation, and training data quality audits for large language model evaluation and computer vision annotation. This is a full-time, remote opportunity aligned with Ottawa talent.

What You Will Do

• Execute senior-level data labeling across text, image, and multimodal tasks with consistent decision-making and high accuracy
• Perform RLHF-style preference labeling/ranking using rubric-based guidelines
• Run prompt evaluation and QA evaluation to identify failure modes and policy gaps
• Audit training data quality via sampling, error categorization, and root-cause analysis
• Ensure annotation guidelines compliance; propose clarifications and edge-case examples
• Support NER/span labeling and content safety labeling as needed
• Resolve disagreements, calibrate annotators, and stabilize inter-annotator agreement
• Document decisions and acceptance criteria for downstream model training teams

Core Workflows

• Gold set creation, consensus labeling, disagreement analysis
• Quality systems: spot checks, batch acceptance, error tracking, corrective actions
• Evaluation datasets for LLMs (rubric scoring, adversarial/safety probes)
• Computer vision annotation (bounding boxes, polygons, segmentation) where applicable

Who You Are

• Experienced in data labeling/annotation with strong edge-case handling and consistency
• Comfortable with RLHF, prompt evaluation, and QA evaluation concepts
• Able to mentor peers and communicate clearly when guidelines conflict with real data