Analytical Evaluator - AI Feedback
$70 per hourMercor
Toronto, ON
- Remote job
About the job
Mercor connects elite creative and technical talent with leading AI research labs. Headquartered in San Francisco, our investors include Benchmark , General Catalyst , Peter Thiel , Adam D'Angelo , Larry Summers , and Jack Dorsey .
Position: Generalist Expert
Type: Contract
Compensation: $70/hour
Location: Remote
Role Responsibilities
- Evaluate AI-generated responses to strengthen reasoning and rigor in model outputs .
- Provide structured written feedback to improve training data quality and downstream performance.
- Develop clear, precise, and well-evidenced written rationales that go beyond surface-level observations.
- Ensure consistent and honest judgment, including critical assessments when warranted.
- Work independently and asynchronously to meet deadlines while improving AI model performance .
Qualifications
Must-Have
- Bachelor's degree from a top-500 globally ranked university preferred.
- Strong analytical and written communication skills.
- Ability to work independently and follow detailed task guidelines.
- Strong critical reading skills with the ability to identify nuance, implicit meaning, and gaps in reasoning.
- Native English fluency required.
Application Process (Takes 20–30 mins to complete)
- Upload resume
- AI interview based on your resume
- Submit form
Resources & Support
- For details about the interview process and platform information, please check:
- For any help or support, reach out to: Show email
PS: Our team reviews applications daily. Please complete your AI interview and application steps to be considered for this opportunity.
Vacancy posted 1 day ago
Similar jobs that could be interesting for youBased on the Analytical Evaluator - AI Feedback in Toronto, ON vacancy
$70 per hour
...technical talent with leading AI research labs. Headquartered in... ...Role Responsibilities Evaluate AI-generated responses to... ...Provide structured written feedback to improve training data quality... ...university preferred. Strong analytical and written communication...SuggestedRemote jobContract workSummer work$100 - $150 per hour
...technical talent with leading AI research labs. Headquartered in... ...A/B test write-ups . Evaluate AI-generated or human-created... ...detailed written justifications for evaluations and scores to maintain... .... Incorporate structured feedback from senior reviewers and iterate...SuggestedFull timeContract workSummer workRemote work$80 per hour
...creative and technical talent with leading AI research labs. Headquartered in San... ...frontier AI coding agents to complete and evaluate complex data engineering tasks. Review... ...involving ETL pipelines , data warehouses , analytics platforms , and distributed data systems...SuggestedFull timeContract workSummer workRemote work- ...Responsibilities Run human and automated evaluations across large question sets and datasets. Statistically measure factual grounding... ...with test automation frameworks and scripting. Strong analytical, problem-solving, communication, collaboration, and decision-making...SuggestedFull time
$80 - $120 per hour
...creative and technical talent with leading AI research labs. Headquartered in San... ...regulatory response with financial-services AI Evaluator Type: Contract Compensation... .... Provide clear, structured written feedback to improve AI model outputs. Apply deep...SuggestedRemote jobContract workSummer workWork at office$100 - $150 per hour
...technical talent with leading AI research labs. Headquartered in... ..., and technical reports. Evaluate AI-generated or human-... ...detailed written justifications for evaluations and scores to maintain... .... Incorporate structured feedback from senior reviewers and iterate...Remote jobContract workSummer work$55 per hour
Our client is sourcing for a Pricing Analytics Manager for a 6-month contract opportunity This... ...price increases and other pricing changes. Evaluate the expected impact of pricing actions on... ...to this position are subject to the use of AI to perform preliminary candidate...Contract workFor contractorsWork at officeLocal area$85 per hour
...Mercor connects elite creative and technical talent with leading AI research labs. Headquartered in San Francisco, our investors... ...Responsibilities Use frontier AI coding agents to complete and evaluate complex infrastructure engineering tasks. Review model-...Full timeContract workSummer workRemote work$80 - $120 per hour
...creative and technical talent with leading AI research labs. Headquartered in San... ...regulatory response with financial-services AI Evaluator Type: Contract Compensation... .... Provide clear, structured written feedback to improve AI model outputs. Apply deep...Remote jobContract workSummer workWork at office$80k - $180k per year
...Zafin is an AI platform company helping regulated institutions modernize how critical... ...scale. What’s the Opportunity? The AI Evaluation Engineer ensures AI agent solutions are accurate... ...across releases, and providing actionable feedback that continuously improves AI agent...Full time$80 - $120 per hour
...connects elite creative and technical talent with leading AI research labs. Headquartered in San Francisco, our investors... ...Jack Dorsey . Position: User/customer research and feedback synthesis Evaluator Type: Contract Compensation: $80–$120/hour...Full timeContract workSummer workWork at officeRemote work$39 per hour
...Mercor connects elite creative and technical talent with leading AI research labs. Headquartered in San Francisco, our investors... ...Commitment: 10+ hours/week Role Responsibilities Evaluate AI model output lyrics , voice generation, and other standards...Remote jobContract workSummer workImmediate start$120 - $170 per hour
...creative and technical talent with leading AI research labs. Headquartered in San... ...and defensible. Incorporate structured feedback from senior reviewers and iterate quickly... ...~ Prior experience with AI training , evaluation, or human-data projects. Application...Remote jobContract workSummer work$70 per hour
...About the job Mercor connects elite creative and technical talent with leading AI research labs. Headquartered in San Francisco our investors include Benchmark General Catalyst Peter Thiel Adam DAngelo Larry Summers and Jack Dorsey . Position: Mathematics...Full timeContract workSummer workImmediate startRemote work$70 - $80 per hour
...Mercor connects elite creative and technical talent with leading AI research labs. Headquartered in San Francisco, our investors... .... ~ Comfortable writing precise technical specifications and evaluation criteria. Start Date ~ Immediate Application...Full timeSummer workFreelanceImmediate startRemote work$80 - $120 per hour
...connects elite creative and technical talent with leading AI research labs. Headquartered in San Francisco, our investors... ...Jack Dorsey . Position: User/customer research and feedback synthesis Evaluator Type: Contract Compensation: $80–$120/hour...Full timeContract workSummer workWork at officeRemote work- ...we need is someone whose entire job is making creators better, reviewing every content submission, giving detailed written and verbal feedback, running training calls, spotting trends, and actively developing creators who produce content that converts and drives GMV. The...Long term contractContract workImmediate startRemote workFlexible hoursWeekend work
$55 - $62 per hour
...Job Title: Community & Customer Feedback Operations Specialist Job... ...ideas. Review, triage, and evaluate customer feedback to determine... ...activities. ~ Experience leveraging AI tools to synthesize... ...Operations teams. ~ Strong analytical, problem-solving, and critical...Remote jobHourly payLong term contractContract work$80 - $150 per hour
...technical talent with leading AI research labs. Headquartered in... ...: MS Excel / Google Sheets Evaluator Type: Contract... ...relevance. Provide structured feedback to enhance document evaluation... ...relevant domain. ~ Strong analytical and written communication skills...Contract workSummer workRemote work$80 - $120 per hour
...creative and technical talent with leading AI research labs. Headquartered in San... ...Position: Customer success / support operations Evaluator Type: Contract... ...errors. Provide clear, structured written feedback. Apply deep subject-matter expertise...Full timeContract workSummer workWork at officeRemote work$100 - $150 per hour
...technical talent with leading AI research labs. Headquartered in... .... Incorporate structured feedback from senior reviewers and... ...supporting proofs of concept, and evaluating product fit and technical risk... ...experience with AI training , evaluation, or human-data projects....Full timeContract workSummer workRemote work$140k - $175k per year
...Summary: Product Analytics is building self-service tools and operating AI agents that influence product development; agents that monitor product health, surface... ...the shared repositories, standards, context, and evaluation tooling our analysts depend on, and you will define...RemplacementFull timeWork at officeLocal areaFlexible hours2 days per week3 days per week$60 - $80 per hour
...creative and technical talent with leading AI research labs. Headquartered in San... ...Position: Business Domain Expert — AI Model Evaluation Pilot (Admin, Marketing, HR, Accounting)... ...parts of the output. Provide structured feedback on the task itself and on the study design...Hourly payFull timeContract workFor contractorsSummer workRemote work$231.2k - $299.2k per year
...take care of and improve your home — from personalized guidance to AI tools and a best-in-class hiring experience. Every day in every... ...you, join us. Imagine what we’ll build together. About the Pro Feedback and Targeting Team At Thumbtack, the Pro Feedback and...Full timeSeasonal workLocal areaFlexible hours- ...is the leading security-first enterprise AI company. We build cutting-edge foundation... ...Berlin and Seoul. Join us! Why this role? Evaluation is critical to making progress in scaling... ...cross-functional teams to translate model feedback into trustworthy, repeatable evaluations....Full timeWork at officeLocal areaRemote workHome office
Cohere
Toronto, ON5 days ago $80 - $120 per hour
...creative and technical talent with leading AI research labs. Headquartered in San... ...Position: IP / trademark / copyright law Evaluator Type: Contract Compensation... ...errors. Provide clear structured written feedback. Work independently and asynchronously...Full timeContract workSummer workWork at officeRemote work$70 - $100 per hour
...creative and technical talent with leading AI research labs. Headquartered in San... ...hours/week Role Responsibilities Evaluate AI-generated office artifacts across Excel... ...outputs using structured rubrics and provide feedback to improve model behavior....Remote jobContract workSummer workWork at officeImmediate start$80 - $120 per hour
...creative and technical talent with leading AI research labs. Headquartered in San... ...Position: Real estate / hospitality / events Evaluator Type: Contract Compensation... .... Provide clear, structured written feedback to improve AI model outputs . Apply...Remote jobContract workSummer workWork at office$80 - $120 per hour
...creative and technical talent with leading AI research labs. Headquartered in San... ...Dorsey . Position: General Sales / GTM Evaluator Type: Contract Compensation... ...errors. Provide clear structured written feedback. Apply deep subject-matter expertise...Full timeContract workSummer workWork at officeRemote work$80 - $120 per hour
...creative and technical talent with leading AI research labs. Headquartered in San... ...Position: Finance operations / audit support Evaluator Type: Contract Compensation... ...errors. Provide clear, structured written feedback. Work independently and...Remote jobContract workSummer workWork at office
Do you want to receive more vacancies?
Subscribe and receive similar vacancies to Analytical Evaluator - AI Feedback. Be the first to apply!
Related searches
- chargé de projet estimateur Toronto, ON
- remote social media evaluator Toronto, ON
- test evaluator Toronto, ON
- admissions evaluator Toronto, ON
- estimateur en bâtiment Toronto, ON
- estimateur automobile Toronto, ON
- estimateur structure Toronto, ON
- évaluateur-estimateur Toronto, ON
- work from home social media evaluator
- chargé de projet estimateur