Remote AI Quality Evaluator - Contractor

YO AI Labs

YO AI Labs is seeking an AI Evaluation Specialist (Contractor) to remotely assess enterprise AI outputs. You will score responses against rubrics, identify issues, and provide actionable feedback to improve AI systems.

No prior AI experience is required, making this an accessible entry point for analytical professionals. The role involves high-volume evaluation, independent work, and collaboration to refine evaluation criteria and best practices within a flexible, remote setup across multiple

#J-18808-Ljbffr

Apply

Vacancy posted 2 hours ago

Similar jobs that could be interesting for youBased on the Remote AI Quality Evaluator - Contractor in Toronto, ON vacancy

AI Product Quality Specialist
$15 - $30 per hour
...Canada Type: Independent Contractor (potential for conversion to full... ..., design, and implement AI strategies that are secure, scalable... ...the person who keeps product quality tight so leadership can stay... ...targeting $15-30/hr. Work is fully remote; US or Canada-based candidates...
For contractors
Remote job
Quality
Hourly pay
Full time
Contract work
Manual labor
Immediate start
Flexible hours
Human Agency Inc
Toronto, ON
25 days ago
Compliance AI Evaluator - Finance
$80 - $120 per hour
...technical talent with leading AI research labs. Headquartered in... ...with financial-services AI Evaluator Type: Contract... ...$120/hour Location: Remote Role Responsibilities... ...artifacts against domain-specific quality rubrics. Identify factual...
Remote work
Quality
Full time
Contract work
Summer work
Work at office
Mercor
Toronto, ON
more than 2 months ago
Data Scientist - AI Evaluation
$100 - $150 per hour
...technical talent with leading AI research labs. Headquartered in... ...–$150/hour Location: Remote Role Responsibilities... ...and A/B test write-ups . Evaluate AI-generated or human-created... ...established criteria to ensure quality and consistency. Provide detailed...
Remote work
Quality
Full time
Contract work
Summer work
Mercor
Toronto, ON
more than 2 months ago
Financial Compliance AI Evaluator
$80 - $120 per hour
...technical talent with leading AI research labs. Headquartered in... ...with financial-services AI Evaluator Type: Contract... ...$120/hour Location: Remote Role Responsibilities... ...artifacts against domain-specific quality rubrics. Identify factual...
Remote work
Quality
Full time
Contract work
Summer work
Work at office
Mercor
Toronto, ON
more than 2 months ago
Procurement Evaluator - Fully Remote
$80 - $120 per hour
...technical talent with leading AI research labs. Headquartered in... .../ vendor management Evaluator Type: Contract... ...$120/hour Location: Remote Role Responsibilities... ...artifacts against domain-specific quality rubrics. Identify factual...
Remote work
Quality
Full time
Contract work
Summer work
Work at office
Mercor
Toronto, ON
more than 2 months ago
Evaluation Specialist - Fully Remote
$80 - $160 per hour
...technical talent with leading AI research labs. Headquartered in... ...$160/hour Location: Remote Role Responsibilities... ...-specific documents to ensure quality and accuracy. Provide structured... ...to contribute to high-impact evaluation projects. Collaborate...
Remote work
Quality
Full time
Contract work
Summer work
Mercor
Toronto, ON
a month ago
DevOps Engineer - AI Model Evaluator
$85 per hour
...creative and technical talent with leading AI research labs. Headquartered in San Francisco... ...: $85/hour Location: Remote Role Responsibilities Use frontier AI coding agents to complete and evaluate complex infrastructure engineering tasks....
Remote work
Full time
Contract work
Summer work
Mercor
Toronto, ON
more than 2 months ago
Data Engineer - AI Model Evaluator
$80 per hour
...creative and technical talent with leading AI research labs. Headquartered in San Francisco... ...: $80/hour Location: Remote Role Responsibilities Use frontier AI coding agents to complete and evaluate complex data engineering tasks. Review...
Remote work
Full time
Contract work
Summer work
Mercor
Toronto, ON
more than 2 months ago
AI Evaluation Specialist - Remote
...AI Evaluation Specialist Role Type: Contractor Location: Remote — US, Canada, UK, Ireland, Australia, New Zealand We are seeking experienced AI Evaluation Specialists to assess and improve the quality of AI assistant outputs for an enterprise AI training project...
For contractors
Remote job
Quality
YO AI Labs
Toronto, ON
20 days ago
Technical Product Manager (AI) - Veeva Quality Cloud
$100k - $150k per year
...manager to design and deliver high-value AI Agent solutions across the Veeva Quality Cloud suite of products. Veeva... ...using AI Assist with AI Agent evaluations Present feature capabilities and... ...with customers and internal teams remotely and across geographies Interviewing...
Remote job
Quality
Full time
Internship
Work at office
Local area
Work from home
Veeva Systems
Toronto, ON
a month ago
Quality Assurance Specialist - Fully Remote
$20 - $50 per hour
...technical talent with leading AI research labs. Headquartered in... ...Dorsey . Position: Quality Analyst Type: Contract... ...–$50/hour Location: Remote Role Responsibilities... ...quality processes, checklists, and evaluation criteria. Collaborate asynchronously...
Remote work
Quality
Full time
Contract work
Summer work
Mercor
Toronto, ON
a month ago
Interim Senior Accountant - Contractor
...issues. Key details Engagement: Contractor, up to 2 days a week (days to be agreed, with... ...possibility of extension Location: Remote, UK-based Start date: December 2026, including... ...Advanced Excel Comfortable using AI tools (Claude) in your daily work, and...
For contractors
Remote job
Daily paid
Full time
Contract work
Interim role
Worldwide
Overseas
Flexible hours
2 days per week
Vira Health
Toronto, ON
18 hours ago
AI Safety Specialist - Fully Remote
$16 - $22 per hour
...creative and technical talent with leading AI research labs. Headquartered in San... ...Compensation: $16–$22/hour Location: Remote Role Responsibilities Red team... ...and bias exploitation. Generate high-quality human data by annotating failures,...
Remote work
Quality
Full time
Contract work
Summer work
Mercor
Toronto, ON
11 days ago
Senior Experience Designer, UI Design (Contractor)
...across Retail, CPG, Sports, and Media with AI-native delivery capability, designing and... ...visit . LOCATION: APPLY is hybrid/remote friendly. The preferred candidate should be... ...current vernacular of AI design tools and evaluate new ones as they emerge. Bring strong...
For contractors
Remote work
Full time
Work at office
Flexible hours
E2x Is Now Part Of Apply Digital
Toronto, ON
28 days ago
Software Engineering Evaluator - Expert
$100 - $150 per hour
...and technical talent with leading AI research labs. Headquartered in San... ...$100–$150/hour Location: Remote Role Responsibilities Evaluate AI-generated artifacts for usability... ...to grade outputs and enhance quality. Work independently and asynchronously...
Remote work
Quality
Contract work
Summer work
Work at office
Mercor
Toronto, ON
a month ago
Data Scientist - AI Evaluation
$100 - $150 per hour
...technical talent with leading AI research labs. Headquartered in... ...–$150/hour Location: Remote Role Responsibilities... ..., and technical reports. Evaluate AI-generated or human-created... ...established criteria to ensure quality and accuracy. Provide detailed...
Remote job
Quality
Contract work
Summer work
Mercor
Toronto, ON
5 days ago
Business Operations Expert - Evaluator
$60 - $80 per hour
...technical talent with leading AI research labs. Headquartered in... ...Domain Expert — AI Model Evaluation Pilot (Admin, Marketing, HR, Accounting... ...–$80/hour Location: Remote Role Responsibilities... ...Compensation & Legal ~ Hourly contractor Application Process (...
For contractors
Remote work
Hourly pay
Full time
Contract work
Summer work
Mercor
Toronto, ON
more than 2 months ago
Frontend Code Evaluation Specialist
$90 per hour
...technical talent with leading AI research labs. Headquartered in... ...$90/hour Location: Remote Role Responsibilities... ...attempts and grade construction quality. Distinguish a replication that... ..., preference-labeling, model-evaluation, or structured code-review work...
Remote work
Quality
Full time
Contract work
Summer work
Local area
Mercor
Toronto, ON
more than 2 months ago
Excel Specialist - Document Evaluator
$80 - $150 per hour
...technical talent with leading AI research labs. Headquartered in... ...: MS Excel / Google Sheets Evaluator Type: Contract... ...$150/hour Location: Remote Role Responsibilities... ...domain-specific documents for quality, accuracy, and relevance....
Remote work
Quality
Full time
Contract work
Summer work
Mercor
Toronto, ON
more than 2 months ago
Adversarial AI Specialist - Fully Remote
$20 - $22 per hour
...creative and technical talent with leading AI research labs. Headquartered in San... ...Compensation: $20–$22/hour Location: Remote Role Responsibilities Red team... ...and bias exploitation. Generate high-quality human data. Annotate failures, classify...
Remote work
Quality
Full time
Contract work
Summer work
Mercor
Toronto, ON
a month ago
Quality Inspector / Coordinator
$60k - $105k per year
...construction projects, distinguished by the high quality of their work, meeting the highest... ...specifications, and standards. Monitor contractor work for compliance with contract specifications... ..., EBC may use artificial intelligence (AI) tools to support the candidate pre-...
For contractors
Quality
Full time
Contract work
Temporary work
For subcontractor
Work at office
Ebc Corp
Toronto, ON
a month ago
Video Evaluation Specialist - Annotator
$15 - $20 per hour
...technical talent with leading AI research labs. Headquartered in... .... Position: Video Evaluation Generalist Type: Contract... ...–$20/hour Location: Remote Commitment: 20 hours/... ...for prompt adherence, motion quality, visual artifacts, and physical...
Remote work
Quality
Full time
Contract work
Summer work
Mercor
Toronto, ON
a month ago
Hindi Music Specialist - Evaluator
$14 - $42 per hour
...technical talent with leading AI research labs. Headquartered in... ...14–$42/hour Location: Remote Duration: Up to 6 months... ...Role Responsibilities Evaluate AI-generated music across... ...genres. Rate against detailed quality standards. Compare AI-generated...
Remote work
Quality
Full time
Contract work
Summer work
Immediate start
Mercor
Toronto, ON
more than 2 months ago
AI Agent Evaluation Engineer - Remote
...Senior Software Engineer Job Type: Contractor (~15 hours/week) Location: Remote Job Summary We are seeking... ...Software Engineers to support an AI training project by creating reinforcement learning environments that evaluate AI models on complex software...
For contractors
Remote job
YO AI Labs
Toronto, ON
more than 2 months ago
Customer Success Evaluator - Expert
$80 - $120 per hour
...technical talent with leading AI research labs. Headquartered in... ...success / support operations Evaluator Type: Contract... ...$120/hour Location: Remote Role Responsibilities... ...artifacts against domain-specific quality rubrics. Identify factual...
Remote work
Quality
Full time
Contract work
Summer work
Work at office
Mercor
Toronto, ON
more than 2 months ago
Bilingual Evaluator - LLM Specialist
$15 - $20 per hour
...creative and technical talent with leading AI research labs. Headquartered in San... ...Compensation: $15–$20/hour Location: Remote Role Responsibilities... ...and external tools . Generate high-quality human evaluation data by identifying response strengths,...
Remote work
Quality
Full time
Contract work
Summer work
Mercor
Toronto, ON
more than 2 months ago
Legal Compliance Expert - Evaluator
$80 - $120 per hour
...technical talent with leading AI research labs. Headquartered in... ...Position: Legal / compliance Evaluator Type: Contract... ...$120/hour Location: Remote Role Responsibilities... ...artifacts against domain-specific quality rubrics. Identify factual...
Remote work
Quality
Full time
Contract work
Summer work
Work at office
Mercor
Toronto, ON
more than 2 months ago
Legal Compliance Evaluator - Expert
$80 - $120 per hour
...technical talent with leading AI research labs. Headquartered in... ...Position: Legal / compliance Evaluator Type: Contract... ...$120/hour Location: Remote Role Responsibilities... ...artifacts against domain-specific quality rubrics. Identify factual...
Remote work
Quality
Full time
Contract work
Summer work
Work at office
Mercor
Toronto, ON
more than 2 months ago
Sales Evaluator - GTM Specialist
$80 - $120 per hour
...technical talent with leading AI research labs. Headquartered in... ...: General Sales / GTM Evaluator Type: Contract... ...$120/hour Location: Remote Role Responsibilities... ...artifacts against domain-specific quality rubrics. Identify factual...
Remote work
Quality
Full time
Contract work
Summer work
Work at office
Flexible hours
Mercor
Toronto, ON
more than 2 months ago
AI Linguistic Evaluator - Marathi
usd16 per hour
...We are looking for AI Linguistic Evaluators to assess how effectively an AI application performs in Marathi... .... Job Type: Freelance Location: Remote from Canada Work Schedule: Flexible Start... ...English skills and a keen eye for language quality? Join a remote AI project where you'll...
Remote work
Quality
Hourly pay
Daily paid
Full time
Freelance
Immediate start
Flexible hours
Rws
Toronto, ON
21 days ago

Do you want to receive more vacancies?

Subscribe and receive similar vacancies to Remote AI Quality Evaluator - Contractor. Be the first to apply!