Analytical Evaluator - AI Feedback
$70 per hourMercor
Toronto, ON
- Remote job
About the job
Mercor connects elite creative and technical talent with leading AI research labs. Headquartered in San Francisco, our investors include Benchmark , General Catalyst , Peter Thiel , Adam D'Angelo , Larry Summers , and Jack Dorsey .
Position: Generalist Expert
Type: Contract
Compensation: $70/hour
Location: Remote
Role Responsibilities
- Evaluate AI-generated responses to strengthen reasoning and rigor in model outputs .
- Provide structured written feedback to improve training data quality and downstream performance.
- Develop clear, precise, and well-evidenced written rationales that go beyond surface-level observations.
- Ensure consistent and honest judgment, including critical assessments when warranted.
- Work independently and asynchronously to meet deadlines while improving AI model performance .
Qualifications
Must-Have
- Bachelor's degree from a top-500 globally ranked university preferred.
- Strong analytical and written communication skills.
- Ability to work independently and follow detailed task guidelines.
- Strong critical reading skills with the ability to identify nuance, implicit meaning, and gaps in reasoning.
- Native English fluency required.
Application Process (Takes 20–30 mins to complete)
- Upload resume
- AI interview based on your resume
- Submit form
Resources & Support
- For details about the interview process and platform information, please check:
- For any help or support, reach out to: Show email
PS: Our team reviews applications daily. Please complete your AI interview and application steps to be considered for this opportunity.
Vacancy posted a month ago
Similar jobs that could be interesting for youBased on the Analytical Evaluator - AI Feedback in Toronto, ON vacancy
$70 per hour
...technical talent with leading AI research labs. Headquartered in... ...Role Responsibilities Evaluate AI-generated responses to... ...Provide structured written feedback to improve training data quality... ...university preferred. Strong analytical and written communication...SuggestedRemote jobContract workSummer work- Cohere is seeking a Senior Research Scientist focused on AI evaluation methods to support our innovative foundation models. This remote or on... .... Collaborate with cross-functional teams to refine model feedback and improve evaluation processes. Your contributions will be pivotal...SuggestedRemote work
usd16 per hour
...We are looking for AI Linguistic Evaluators to assess how effectively an AI application performs in Marathi for a short collaboration. Job... ..., and cultural appropriateness. - Providing structured evaluations and feedback following the project instructions and quality standards...SuggestedHourly payDaily paidFull timeFreelanceImmediate startRemote workFlexible hours$155k - $215k per year
...role.Job Description:The AVP, AI Product leads a material enterprise... ..., design, build / blend, evaluation, control readiness, launch, adoption... ...use, enable users, gather feedback and sustain client and... ...technology delivery, AI/data/analytics, platform leadership or transformation...SuggestedWork from home$70 per hour
...Mindrift connects specialists with project-based AI opportunities for leading tech companies, focused on testing, evaluating, and improving AI systems. Participation is... ...evaluate clinical EHR vignettes for a medical AI evaluation program. General and internal medicine are our...SuggestedHourly payPermanent employmentPart time10 hours per week$20 - $160 per hour
...creative and technical talent with leading AI research labs. Headquartered in San... ...Annotator — Health AI Conversation Quality Evaluation Type: Contract Compensation... ...Preferred Experience in annotation, human-feedback, evaluation, or content review....Remote jobContract workSummer work$102.7k - $137k per year
...Position Title: Engineering Manager, Data AI & Analytics Position Type: Regular - Full-Time Requisition ID: 42289 In every role,... ...scalability, reliability, and deployment efficiency Continuously evaluate emerging technologies and implement improvements across data...Full timeWork from home$140k - $175k per year
...Summary Product Analytics is building self-service tools and operating AI agents that influence product development; agents that monitor product health, surface... ...the shared repositories, standards, context, and evaluation tooling our analysts depend on, and you will define...Work at officeLocal areaFlexible hours2 days per week3 days per week$80 - $120 per hour
...creative and technical talent with leading AI research labs. Headquartered in San... ...regulatory response with financial-services AI Evaluator Type: Contract Compensation... .... Provide clear, structured written feedback to improve AI model outputs. Apply deep...Full timeContract workSummer workWork at officeRemote work$80 - $120 per hour
...creative and technical talent with leading AI research labs. Headquartered in San... ...regulatory response with financial-services AI Evaluator Type: Contract Compensation... .... Provide clear, structured written feedback. Apply deep subject-matter expertise...Full timeContract workSummer workWork at officeRemote work- ...consumer experiences, from mobile and web applications to advanced analytics and machine learning, to content and engagement management... ...digital experiences and client partnerships. As a QA / AI Evaluation Engineer, you will join a highly motivated and experienced team...Full timeImmediate start
$100 - $150 per hour
...technical talent with leading AI research labs. Headquartered in... ..., and technical reports. Evaluate AI-generated or human-... ...detailed written justifications for evaluations and scores to maintain... .... Incorporate structured feedback from senior reviewers and iterate...Remote jobContract workSummer work$100 - $150 per hour
...technical talent with leading AI research labs. Headquartered in... ...A/B test write-ups . Evaluate AI-generated or human-created... ...detailed written justifications for evaluations and scores to maintain... .... Incorporate structured feedback from senior reviewers and iterate...Full timeContract workSummer workRemote work$80 per hour
...creative and technical talent with leading AI research labs. Headquartered in San... ...frontier AI coding agents to complete and evaluate complex data engineering tasks. Review... ...involving ETL pipelines , data warehouses , analytics platforms , and distributed data systems...Full timeContract workSummer workRemote work$70 per hour
...About the job Mercor connects elite creative and technical talent with leading AI research labs. Headquartered in San Francisco, our investors include Benchmark , General Catalyst , Peter Thiel , Adam D'Angelo , Larry Summers , and Jack Dorsey . Position...Remote jobContract workSummer workImmediate start$80 - $120 per hour
...creative and technical talent with leading AI research labs. Headquartered in San... ...Position: Software / AI / IT / data Evaluator Type: Contract Compensation... ...outputs. Provide clear, structured written feedback to improve quality. Work independently...Remote jobContract workSummer workWork at office$82.1k - $132.1k per year
...Job Description Product Analytics is building self-service tools and operating AI agents that influence product development... ...repositories, standards, and evaluation tooling the AI Engineering Lead... ...interfaces, semantic layers, and feedback loops that let those systems...RemplacementFull timeWork at officeLocal areaFlexible hours$80 - $120 per hour
...creative and technical talent with leading AI research labs. Headquartered in San... ...regulatory response with financial-services AI Evaluator Type: Contract Compensation... .... Provide clear, structured written feedback to improve AI model outputs. Apply deep...Remote jobContract workSummer workWork at office- ...Data & Analytics Leader - BFSI, AI Transformation The Opportunity We are seeking an experienced Data & Analytics Leader to help shape and grow enterprise transformation programs across the financial services...
- ...-winning culture. What we offer Reporting to the Chief Technology & Information Officer (CTIO), the Vice President, Data Analytics & AI is the executive leader responsible for setting the enterprise vision, strategy, and execution for Mattamy’s data, analytics, and...Long term contractFull timeInternshipWork at office
$85 per hour
...Mercor connects elite creative and technical talent with leading AI research labs. Headquartered in San Francisco, our investors... ...Responsibilities Use frontier AI coding agents to complete and evaluate complex infrastructure engineering tasks. Review model-...Full timeContract workSummer workRemote work- ...Program Manager will lead cybersecurity research, engineering, product evaluation, and emerging-technology initiatives from hypothesis through... ...teams, vendors, startups, and leadership to drive programs across AI Security, Agentic AI, and other technologies. #J-18808-Ljbffr
$39 per hour
...Mercor connects elite creative and technical talent with leading AI research labs. Headquartered in San Francisco, our investors... ...Commitment: 10+ hours/week Role Responsibilities Evaluate AI model output lyrics , voice generation, and other standards...Remote jobContract workSummer workImmediate start$120 - $170 per hour
...creative and technical talent with leading AI research labs. Headquartered in San... ...and defensible. Incorporate structured feedback from senior reviewers and iterate quickly... ...~ Prior experience with AI training , evaluation, or human-data projects. Application...Remote jobContract workSummer work$80 - $120 per hour
...connects elite creative and technical talent with leading AI research labs. Headquartered in San Francisco, our investors... ...Jack Dorsey . Position: User/customer research and feedback synthesis Evaluator Type: Contract Compensation: $80–$120/hour...Full timeContract workSummer workWork at officeRemote work$140k - $175k per year
...Summary: Product Analytics is building self-service tools and operating AI agents that influence product development; agents that monitor product health, surface... ...the shared repositories, standards, context, and evaluation tooling our analysts depend on, and you will define...RemplacementFull timeWork at officeLocal areaFlexible hours2 days per week3 days per week$60 - $100 per hour
...creative and technical talent with leading AI research labs. Headquartered in San... ...Position: Marketing Expert — Marketing Analytics (AI Agent Environments) Type: Contract... ...them against established standards. Evaluate AI-generated content in Marketing Analytics...Remote jobContract workSummer work$5000 per month
...Lightspeed's Data Office is hiring a Senior Analytics Engineer to design and own the data... ...pragmatic, shipping incremental value, gathering feedback, and iterating as you go Turn ambiguous... ...notes that power both human reviewers and AI/LLM-assisted workflows Diagnose root...Full timeWork at officeImmediate startRemote workFlexible hours- ...Data & Analytics Solutions Architect - AI platforms, AWS, Databricks The Opportunity We are hiring a senior Data & Analytics Solutions Architect to lead the design of enterprise data, analytics, and AI platforms using AWS and Databricks. This role combines hands-on...
- ...for QuickBooks. This role defines the data-science agenda across markets, partnering with product leadership to scale causal evidence and AI-native decision systems. The position is based in Toronto with a hybrid work model, three days in the office. We expect 10+ years in...Work at office
Do you want to receive more vacancies?
Subscribe and receive similar vacancies to Analytical Evaluator - AI Feedback. Be the first to apply!
Related searches
- chargé de projet estimateur Toronto, ON
- estimateur en bâtiment Toronto, ON
- estimateur industriel Toronto, ON
- estimateur structure Toronto, ON
- test evaluator Toronto, ON
- remote social media evaluator Toronto, ON
- admissions evaluator Toronto, ON
- évaluateur-estimateur Toronto, ON
- estimateur automobile Toronto, ON
- ai evaluator