Remote LLM & GenAI Quality Evaluator
Lifted (an Upwork Company)
Lifted (an Upwork Company) is seeking experienced evaluators to assess model-generated content across text, images, audio, video, and HTML widgets. You will use defined rubrics to judge factuality, consistency, and aesthetics, working with detailed project guidelines.
The role is flexible and remote, with workload varying weekly. A Bachelor's degree and 3+ years of hands-on LLM/GenAI data evaluation are required, and independent research using trusted sources is expected.
#J-18808-LjbffrVacancy posted 13 hours ago
Similar jobs that could be interesting for youBased on the Remote LLM & GenAI Quality Evaluator in Toronto, ON vacancy
$15 - $20 per hour
...Contract Compensation: $15–$20/hour Location: Remote Role Responsibilities Conduct fact-checking... ...public sources and external tools . Generate high-quality human evaluation data by identifying response strengths, areas for improvement...Remote workQualityFull timeContract workSummer workMercor
Toronto, ONmore than 2 months ago$100 per hour
...Type: Contract Compensation: $100–$500/hour Location: Remote Role Responsibilities Build and operate LLM agents in production environments. Evaluate agent performance and determine improvements after changes. Manage long-running...Remote workFull timeContract workSummer workMercor
Toronto, ONmore than 2 months ago$80 - $120 per hour
...Procurement / vendor management Evaluator Type: Contract... ...120/hour Location: Remote Role Responsibilities... ...artifacts against domain-specific quality rubrics. Identify factual... ...while ensuring high-quality evaluations. Qualifications Must...Remote workQualityFull timeContract workSummer workWork at officeMercor
Toronto, ONa month ago$80 - $160 per hour
...Compensation: $80–$160/hour Location: Remote Role Responsibilities Review... ...domain-specific documents to ensure quality and accuracy. Provide structured... ...analytical skills to contribute to high-impact evaluation projects. Collaborate with teams to...Remote workQualityFull timeContract workSummer workMercor
Toronto, ONa month ago- ...Software Development Engineer in Test – GenAI / LLM Role Description Work Arrangement: 1 day onsite per week Experience Required... ...gaps. Perform chatbot conversation analysis and recommend quality improvements. Develop and maintain test automation for...QualityPermanent employment1 day per week
$20 - $50 per hour
...Summers , and Jack Dorsey . Position: Quality Analyst Type: Contract... ...Compensation: $20–$50/hour Location: Remote Role Responsibilities Review... ...quality processes, checklists, and evaluation criteria. Collaborate asynchronously...Remote workQualityFull timeContract workSummer workMercor
Toronto, ONa month ago$50 - $60 per hour
...Compensation: $50–$60/hour Location: Remote Role Responsibilities Track... ...Google Sheets . Review annotator quality to ensure high standards. Handle... ...Apply an analytical mindset to improve LLM training projects . Own unfamiliar problems...Remote workQualityFull timeContract workSummer workMercor
Toronto, ONmore than 2 months ago$100 - $150 per hour
...Compensation: $100–$150/hour Location: Remote Role Responsibilities Evaluate AI-generated artifacts for usability in a software... ...subject-matter expertise to grade outputs and enhance quality. Work independently and asynchronously to meet...Remote workQualityContract workSummer workWork at officeMercor
Toronto, ONa month ago- ...Kake is a remote-first company and a people-first global community of senior engineers... ...thrive (and love eating kake!). Senior AI/LLM Engineers with strong Python experience... ...tradeoffs around cost, latency, and quality. Evaluate model outputs systematically, building tooling...Remote workQualityFlexible hours
$80 - $150 per hour
...Position: MS Excel / Google Sheets Evaluator Type: Contract... ...150/hour Location: Remote Role Responsibilities... ...domain-specific documents for quality, accuracy, and relevance.... ...feedback to enhance document evaluation processes. Utilize MS Excel...Remote workQualityFull timeContract workSummer workMercor
Toronto, ONmore than 2 months ago$90 per hour
...Compensation: $90/hour Location: Remote Role Responsibilities Render... ...of both attempts and grade construction quality. Distinguish a replication that is... ...Prior RLHF , preference-labeling, model-evaluation, or structured code-review work. Pixel...Remote workQualityFull timeContract workSummer workLocal areaMercor
Toronto, ONmore than 2 months ago$80 - $120 per hour
...Public-sector procurement / RFI response Evaluator Type: Contract Compensation: $80–$120/hour Location: Remote Role Responsibilities Evaluate... ...generated artifacts against domain-specific quality rubrics. Identify factual, aesthetic,...Remote workQualityFull timeContract workSummer workWork at officeFlexible hoursMercor
Toronto, ONmore than 2 months ago$14 - $42 per hour
...Compensation: $14–$42/hour Location: Remote Duration: Up to 6 months... ...hours/week Role Responsibilities Evaluate AI-generated music across various genres. Rate against detailed quality standards. Compare AI-generated lyrics with...Remote workQualityFull timeContract workSummer workImmediate startMercor
Toronto, ONmore than 2 months ago$15 - $20 per hour
...and Jack Dorsey . Position: Video Evaluation Generalist Type: Contract... ...Compensation: $15–$20/hour Location: Remote Commitment: 20 hours/week... ...provided rubric for prompt adherence, motion quality, visual artifacts, and physical plausibility...Remote workQualityFull timeContract workSummer workMercor
Toronto, ONa month ago$80 - $120 per hour
...Dorsey . Position: General Sales / GTM Evaluator Type: Contract Compensation: $80–$120/hour Location: Remote Role Responsibilities Evaluate... ...artifacts against domain-specific quality rubrics. Identify factual, aesthetic,...Remote workQualityFull timeContract workSummer workWork at officeFlexible hoursMercor
Toronto, ONmore than 2 months ago$100 - $150 per hour
...150/hour Location: Remote Role Responsibilities... ...A/B test write-ups . Evaluate AI-generated or human-created... ...established criteria to ensure quality and consistency. Provide... ...detailed written justifications for evaluations and scores to maintain...Remote workQualityFull timeContract workSummer workMercor
Toronto, ONmore than 2 months ago$80 - $120 per hour
...regulatory response with financial-services AI Evaluator Type: Contract Compensation: $80–$120/hour Location: Remote Role Responsibilities... ...generated artifacts against domain-specific quality rubrics. Identify factual, aesthetic,...Remote workQualityFull timeContract workSummer workWork at officeMercor
Toronto, ONmore than 2 months ago$80 - $120 per hour
...Dorsey . Position: Legal / compliance Evaluator Type: Contract Compensation: $80–$120/hour Location: Remote Role Responsibilities Evaluate... ...artifacts against domain-specific quality rubrics. Identify factual, aesthetic,...Remote workQualityFull timeContract workSummer workWork at officeMercor
Toronto, ONmore than 2 months ago$80 - $120 per hour
...Dorsey . Position: Legal / compliance Evaluator Type: Contract Compensation: $80–$120/hour Location: Remote Role Responsibilities... ...generated artifacts against domain-specific quality rubrics. Identify factual, aesthetic,...Remote workQualityFull timeContract workSummer workWork at officeMercor
Toronto, ONmore than 2 months ago$80 - $120 per hour
...Position: Government / public administration Evaluator Type: Contract Compensation: $80–$120/hour Location: Remote Role Responsibilities Evaluate... ...artifacts against domain-specific quality rubrics. Identify factual, aesthetic,...Remote jobQualityContract workSummer workWork at office$80 - $120 per hour
...Adam D'Angelo , Larry Summers , and Jack Dorsey . Position: Data quality / CRM operations Evaluator Type: Contract Compensation: $80–$120/hour Location: Remote Role Responsibilities Evaluate AI-generated artifacts against...Remote jobQualityContract workSummer workWork at office$100 - $150 per hour
...Compensation: $100–$150/hour Location: Remote Role Responsibilities Design... ..., supporting proofs of concept, and evaluating product fit and technical risk. ~... ...~ Prior experience with AI training , evaluation, or human-data projects. Application...Remote workFull timeContract workSummer workMercor
Toronto, ONmore than 2 months ago$70 - $100 per hour
...Suite Experts Type: Contract Compensation: $70–$100/hour Location: Remote Commitment: 20–40 hours/week Role Responsibilities Evaluate AI-generated office artifacts in Excel , Word , and PowerPoint formats. Perform...Remote workFull timeContract workSummer workWork at officeImmediate startMercor
Toronto, ONmore than 2 months ago$25 - $45 per hour
...Compensation: $25–$45/hour Location: Remote Role Responsibilities Evaluate AI-generated written conversations using predefined... ...rubrics to enhance performance. Draft and refine evaluation rubrics for generalist writing tasks. Apply structured...Remote workFull timeContract workSummer workMercor
Toronto, ONmore than 2 months ago$60 - $80 per hour
...and Jack Dorsey . Position: Business Domain Expert — AI Model Evaluation Pilot (Admin, Marketing, HR, Accounting) Type: Contract Compensation: $60–$80/hour Location: Remote Role Responsibilities Sign an NDA before receiving any...Remote workHourly payFull timeContract workFor contractorsSummer workMercor
Toronto, ONmore than 2 months ago$70k - $73.85k per year
...is for an existing vacancy. Objective: The Research and Evaluation Specialist will lead the analysis and development of sector-based... ...within sector portfolio. ~ Monitors and evaluates the quality and integrity of data within sector to inform evaluation methodologies...QualityFull timeInternshipOfifc
Toronto, ONmore than 2 months ago$15 - $30 per hour
...work and one that's noise. You'll be the person who keeps product quality tight so leadership can stay focused on building what's next.... ...commensurate with experience. We're targeting $15-30/hr. Work is fully remote; US or Canada-based candidates only. Hours are flexible but...Remote jobQualityHourly payFull timeContract workFor contractorsManual laborImmediate startFlexible hoursHuman Agency Inc
Toronto, ON23 days ago$80 per hour
...Experience) Type: Contract Compensation: $80/hour Location: Remote Role Responsibilities Use frontier AI coding agents to complete and evaluate complex data engineering tasks. Review model-generated implementations involving...Remote workFull timeContract workSummer workMercor
Toronto, ONmore than 2 months ago$85 per hour
...Experience) Type: Contract Compensation: $85/hour Location: Remote Role Responsibilities Use frontier AI coding agents to complete and evaluate complex infrastructure engineering tasks. Review model-generated implementations...Remote workFull timeContract workSummer workMercor
Toronto, ONmore than 2 months ago$130k - $180k per year
...the highest standards of technical excellence. This commitment to quality is evident not only in our work but also in how we support and... ...embeddings, fine-tuning) ~ Microsoft Foundry for enterprise-grade LLM lifecycle management ~ LangChain and LangGraph for agent-...Quality
Do you want to receive more vacancies?
Subscribe and receive similar vacancies to Remote LLM & GenAI Quality Evaluator. Be the first to apply!
Related searches
- estimateur structure Toronto, ON
- admissions evaluator Toronto, ON
- chargé de projet estimateur Toronto, ON
- évaluateur-estimateur Toronto, ON
- test evaluator Toronto, ON
- estimateur automobile Toronto, ON
- estimateur - dessinateur Toronto, ON
- remote social media evaluator Toronto, ON
- estimateur en bâtiment Toronto, ON
- estimateur industriel Toronto, ON