AI Evaluator - Software Expert
$80 - $120 per hourMercor
Toronto, ON
- Remote job
About the job
Mercor connects elite creative and technical talent with leading AI research labs. Headquartered in San Francisco, our investors include Benchmark , General Catalyst , Peter Thiel , Adam D'Angelo , Larry Summers , and Jack Dorsey .
Position: Software / AI / IT / data Evaluator
Type: Contract
Compensation: $80–$120/hour
Location: Remote
Role Responsibilities
- Evaluate AI-generated artifacts against domain-specific quality rubrics.
- Identify factual, aesthetic, and presentation errors in documents, spreadsheets, and slide decks .
- Provide clear, structured written feedback to improve AI model outputs .
- Apply deep subject-matter expertise to grade outputs for accuracy and rigor.
- Work independently and asynchronously to meet deadlines.
Qualifications
Must-Have
- 5+ years of relevant professional experience in Software / AI / IT / data .
- Native or professional fluency in English .
- Highly proficient in Microsoft Office and Google Workspace , especially Slides .
Preferred
- Advanced degree ( Master's or higher ) from a reputable institution.
Application Process (Takes 20–30 mins to complete)
- Upload resume
- AI interview based on your resume
- Submit form
Resources & Support
- For details about the interview process and platform information, please check:
- For any help or support, reach out to: Show email
PS: Our team reviews applications daily. Please complete your AI interview and application steps to be considered for this opportunity.
Vacancy posted more than 2 months ago
Similar jobs that could be interesting for youBased on the AI Evaluator - Software Expert in Toronto, ON vacancy
$80 - $120 per hour
...connects elite creative and technical talent with leading AI research labs. Headquartered in San Francisco, our investors... ...Angelo , Larry Summers , and Jack Dorsey . Position: Software / AI / IT / data Evaluator Type: Contract Compensation: $80–$120/...SoftwareRemote jobContract workSummer workWork at office$100 - $150 per hour
...creative and technical talent with leading AI research labs. Headquartered in San... ..., and Jack Dorsey . Position: Software Engineer Expert Type: Contract Compensation... ...Role Responsibilities Evaluate AI-generated artifacts for usability...SoftwareRemote jobContract workSummer workWork at office$70 per hour
...creative and technical talent with leading AI research labs. Headquartered in San... .... Position: Mathematics PhD Coding Experts Type: Contract Compensation... ...-reviewed journals. Prior scientific software or research engineering experience. Start...SoftwareRemote jobContract workSummer workImmediate start$120 - $170 per hour
...creative and technical talent with leading AI research labs. Headquartered in San... ...Jack Dorsey . Position: Data Science Expert Type: Contract Compensation... ...~ Prior experience with AI training , evaluation, or human-data projects. Application...SuggestedRemote jobContract workSummer work$30 per hour
...creative and technical talent with leading AI research labs. Headquartered in San... ...Dorsey . Position: Hindi Linguistic Experts Type: Contract Compensation... ...Remote Role Responsibilities Evaluate the quality and accuracy of LLM-generated...SuggestedRemote jobContract workSummer work$150 per hour
...creative and technical talent with leading AI research labs. Headquartered in San... ...Jack Dorsey . Position: Healthcare Expert Type: Contract Compensation... ...hours/week Role Responsibilities Evaluate and compare AI-generated responses to challenging...Remote jobContract workSummer workImmediate startFlexible hours$100 - $150 per hour
...creative and technical talent with leading AI research labs. Headquartered in San... ...Jack Dorsey . Position: Data Science Expert Type: Contract Compensation... ...Remote Role Responsibilities Evaluate AI-generated artifacts for usability in...Remote jobContract workSummer workWork at office$50 per hour
...technical talent with leading AI research labs. Headquartered in... ...Language and Literature Expert Type: Contract... ...Role Responsibilities Evaluate the quality and accuracy of LLM... ...linguistic annotation, language evaluation, or NLP data work in English...Remote jobContract workSummer work$100 - $150 per hour
...creative and technical talent with leading AI research labs. Headquartered in San... ...pipelines , and technical reports. Evaluate AI-generated or human-created work against... ...Provide detailed written justifications for evaluations and scores to maintain transparency....Remote jobContract workSummer work$100k - $145k per year
...Job Description: What if the AI agents you help build empower... ...to agent observability and evaluation practices (tracing, automated... ...architecting and shipping production software systems, including hands-on... ...Department at HR.Leave-Expert@thomsonreuters.com . Disability...SoftwareFull timeInternshipWork at officeLocal areaFlexible hours$100 per hour
...Mindrift connects specialists with project-based AI opportunities for leading tech companies, focused on testing, evaluating, and improving AI systems. Participation is... ...evaluate clinical EHR vignettes for a medical AI evaluation program. General and internal medicine are our...Hourly payPermanent employmentPart time10 hours per week$140k - $175k per year
...What if the AI agents you architect empower legal professionals... ...Design agent observability and evaluation practices (tracing, automated... ...architecting and shipping production software systems, including hands-on... ...Department at HR.Leave-Expert@thomsonreuters.com . Disability...SoftwareFull timeInternshipWork at officeLocal areaFlexible hoursShift work$140k - $175k per year
...future of legal research with AI? Join our Global Westlaw Engineering... ...and technical strategy – Evaluate and drive adoption of emerging... ...; mentor senior and junior software engineers; champion MLOps and... ...deployment, and metrics. ~ Expert-level proficiency in Python or...SoftwareFull timeWork at officeLocal areaWorldwideFlexible hours$100 - $150 per hour
...technical talent with leading AI research labs. Headquartered in... ...Position: Sales Engineering Expert Type: Contract... ...supporting proofs of concept, and evaluating product fit and technical risk... ...with technically complex B2B software — SaaS , cloud infrastructure...SoftwareRemote jobContract workSummer work$80 - $120 per hour
...creative and technical talent with leading AI research labs. Headquartered in San... ...Position: Privacy / regulatory compliance Evaluator Type: Contract Compensation... ...deadlines while ensuring high-quality evaluations. Collaborate with AI research teams...Remote jobContract workSummer workWork at office- ...Thomson Reuters is looking for a Staff Software Engineer, AI Product Engineering to help shape and... ...across multiple teams or workstreams. Evaluate AI behavior in production and build... ...Human Resources Department at HR.Leave-Expert@thomsonreuters.com . Disability accommodations...SoftwareFull timeWork at officeLocal areaFlexible hours
$80 - $120 per hour
...Mercor connects elite creative and technical talent with leading AI research labs. Headquartered in San Francisco, our investors... ...Jack Dorsey . Position: Finance operations / audit support Evaluator Type: Contract Compensation: $80–$120/hour...Remote jobContract workSummer workWork at office$50 - $60 per hour
...creative and technical talent with leading AI research labs. Headquartered in San... ...Summers , and Jack Dorsey . Position: Software Developers (Must own MacBook) Type:... ...Complete a brief paid assessment to evaluate your fit for the role. Resources & Support...SoftwareRemote jobFull timeContract workSummer workInternshipImmediate startusd16 per hour
...We are looking for AI Linguistic Evaluators to assess how effectively an AI application performs in Marathi for a short collaboration. Job Type... ...naturalness, and cultural appropriateness. - Providing structured evaluations and feedback following the project instructions and quality...Hourly payDaily paidFull timeFreelanceImmediate startRemote workFlexible hours$170k per year
...Role: Software Engineering Expert (EE) Location: Toronto, ON Rates: up to $170k per year Structure... ...being tied to one language/framework. 3. AI + Technology Adoption Practical... ...decision support. Demonstrated ability to evaluate, introduce and drive adoption of new...SoftwareFull timeContract work$80 - $120 per hour
...creative and technical talent with leading AI research labs. Headquartered in San... ...Position: Training / onboarding / L&D Evaluator Type: Contract Compensation... ...to meet deadlines and ensure high-quality evaluations. Qualifications Must-Have...Remote jobContract workSummer workWork at office$140k - $175k per year
...with speed and certainty. As a Staff Software Engineer – AI on the Confirmation Product... ...workflows, safety controls, guardrails, evaluation, and resilient error handling around third... ...Human Resources Department at HR.Leave-Expert@thomsonreuters.com . Disability accommodations...SoftwareLong term contractFull timeWork at officeLocal areaFlexible hours$80 - $120 per hour
...Mercor connects elite creative and technical talent with leading AI research labs. Headquartered in San Francisco, our investors... ...Jack Dorsey . Position: Real estate / hospitality / events Evaluator Type: Contract Compensation: $80–$120/hour...Remote jobContract workSummer workWork at office$80 - $120 per hour
...Mercor connects elite creative and technical talent with leading AI research labs. Headquartered in San Francisco, our investors... .... Position: Investor materials / fundraising / pitchbook Evaluator Type: Contract Compensation: $80–$120/hour...Remote jobContract workSummer workWork at office$80 - $150 per hour
...creative and technical talent with leading AI research labs. Headquartered in San... ...and Jack Dorsey . Position: Design Expert Type: Contract Compensation... ...~ Prior experience with AI training , evaluation, or human-data projects. Application...Remote jobContract workSummer work$80 - $120 per hour
...Mercor connects elite creative and technical talent with leading AI research labs. Headquartered in San Francisco, our investors... ...Jack Dorsey . Position: Biology / environmental science Evaluator Type: Contract Compensation: $80–$120/hour...Remote jobContract workSummer workWork at office$60 - $80 per hour
...connects elite creative and technical talent with leading AI research labs. Headquartered in San Francisco, our investors... ...Summers , and Jack Dorsey . Position: Business Domain Expert — AI Model Evaluation Pilot (Admin, Marketing, HR, Accounting) Type:...Remote jobHourly payContract workFor contractorsSummer work$50 - $70 per hour
...creative and technical talent with leading AI research labs. Headquartered in San... ...Jack Dorsey . Position: Generalist Expert (US/Canada) Type: Contract... ...Role Responsibilities Review and evaluate documents, slides, spreadsheets, and similar...Remote jobContract workSummer workFlexible hours$80 - $120 per hour
...Mercor connects elite creative and technical talent with leading AI research labs. Headquartered in San Francisco, our investors... ...Summers , and Jack Dorsey . Position: Healthcare / clinical Evaluator Type: Contract Compensation: $80–$120/hour...Remote jobContract workSummer workWork at office$100 - $150 per hour
...creative and technical talent with leading AI research labs. Headquartered in San... ...Jack Dorsey . Position: B2B Sales Expert Type: Contract Compensation... ...~ Prior experience with AI training , evaluation, or human-data projects. Application...Remote jobContract workSummer work
Do you want to receive more vacancies?
Subscribe and receive similar vacancies to AI Evaluator - Software Expert. Be the first to apply!
Related searches
- test evaluator Toronto, ON
- estimateur structure Toronto, ON
- estimateur en bâtiment Toronto, ON
- remote social media evaluator Toronto, ON
- évaluateur-estimateur Toronto, ON
- admissions evaluator Toronto, ON
- estimateur - dessinateur Toronto, ON
- estimateur industriel Toronto, ON
- chargé de projet estimateur Toronto, ON
- estimateur automobile Toronto, ON