AI Evaluator - Software Expert
$80 - $120 per hourMercor
Toronto, ON
- Remote job
About the job
Mercor connects elite creative and technical talent with leading AI research labs. Headquartered in San Francisco, our investors include Benchmark , General Catalyst , Peter Thiel , Adam D'Angelo , Larry Summers , and Jack Dorsey .
Position: Software / AI / IT / data Evaluator
Type: Contract
Compensation: $80–$120/hour
Location: Remote
Role Responsibilities
- Evaluate AI-generated artifacts against domain-specific quality rubrics.
- Identify factual, aesthetic, and presentation errors in documents, spreadsheets, and slide decks .
- Provide clear, structured written feedback to improve AI model outputs .
- Apply deep subject-matter expertise to grade outputs for accuracy and rigor.
- Work independently and asynchronously to meet deadlines.
Qualifications
Must-Have
- 5+ years of relevant professional experience in Software / AI / IT / data .
- Native or professional fluency in English .
- Highly proficient in Microsoft Office and Google Workspace , especially Slides .
Preferred
- Advanced degree (Master's or higher) from a reputable institution.
Application Process (Takes 20–30 mins to complete)
- Upload resume
- AI interview based on your resume
- Submit form
Resources & Support
- For details about the interview process and platform information, please check:
- For any help or support, reach out to: Show email
PS: Our team reviews applications daily. Please complete your AI interview and application steps to be considered for this opportunity.
Vacancy posted a month ago
Similar jobs that could be interesting for youBased on the AI Evaluator - Software Expert in Toronto, ON vacancy
$80k - $180k per year
...Zafin is an AI platform company helping regulated institutions modernize how critical... ...scale. What’s the Opportunity? The AI Evaluation Engineer ensures AI agent solutions are accurate... ...3–10+ years of relevant experience in software quality engineering, AI evaluation, AI...SoftwareFull time$60 - $75 per hour
...creative and technical talent with leading AI research labs. Headquartered in San... ..., and Jack Dorsey . Position: Software & Data Science Expert Type: Contract... ...reasoning over technical documentation. Evaluate AI ability to follow precise instructions...SoftwareRemote jobContract workSummer work$80 - $120 per hour
...Mercor connects elite creative and technical talent with leading AI research labs. Headquartered in San Francisco, our investors... ...Summers , and Jack Dorsey . Position: People ops / recruiting Evaluator Type: Contract Compensation: $80–$120/hour...SuggestedRemote jobContract workSummer workWork at office$80 - $120 per hour
...Mercor connects elite creative and technical talent with leading AI research labs. Headquartered in San Francisco, our investors... ...Dorsey . Position: Market research / competitive intelligence Evaluator Type: Contract Compensation: $80–$120/hour...SuggestedRemote jobContract workSummer workWork at officeFlexible hours$80 - $120 per hour
...Mercor connects elite creative and technical talent with leading AI research labs. Headquartered in San Francisco, our investors... ...Jack Dorsey . Position: Biology / environmental science Evaluator Type: Contract Compensation: $80–$120/hour...SuggestedRemote jobContract workSummer workWork at office$80 - $120 per hour
...creative and technical talent with leading AI research labs. Headquartered in San... ...Position: Training / onboarding / L&D Evaluator Type: Contract Compensation... ...feedback. Collaborate with subject matter experts to ensure consistency and quality. Work...Remote jobContract workSummer workWork at office$149k - $168k per year
...dealerships nationwide, delivering cutting-edge software tools to simplify operations, boost... ...We are looking for a hands-on Engineering Expert who will be: setting technical direction across... ...of progression. Help teams adopt AI-assisted engineering safely across coding,...SoftwareInternship$15 - $20 per hour
...Mercor connects elite creative and technical talent with leading AI research labs. Headquartered in San Francisco, our investors... ...sources and external tools . Generate high-quality human evaluation data by identifying response strengths, areas for improvement, and...Remote jobContract workSummer work$80 - $120 per hour
...Mercor connects elite creative and technical talent with leading AI research labs. Headquartered in San Francisco, our investors... ...Summers , and Jack Dorsey . Position: Cybersecurity / IT GRC Evaluator Type: Contract Compensation: $80–$120/hour...Remote jobContract workSummer workWork at office$80 - $120 per hour
...creative and technical talent with leading AI research labs. Headquartered in San... ...Position: Humanities / arts / culture Evaluator Type: Contract Compensation... ...to meet deadlines and ensure high-quality evaluations. Qualifications Must-Have...Remote jobContract workSummer workWork at office$80 - $120 per hour
...creative and technical talent with leading AI research labs. Headquartered in San... ...Operations / inventory / capacity planning Evaluator Type: Contract Compensation... ...meet deadlines and ensure high-quality evaluations. Qualifications Must-Have...Remote jobContract workSummer workWork at office- ...curriculum, developed in partnership with Nebius AI. Our team is fully remote and globally... ...of expertise We will bring on three Experts , each focused on one of the following... ...you REQUIREMENTS ~5+ years as a senior software/systems architect, platform engineer, or senior...SoftwareWork at officeRemote workFlexible hours
$70 per hour
...creative and technical talent with leading AI research labs. Headquartered in San... ...Jack Dorsey . Position: Generalist Expert Type: Contract Compensation... ...Remote Role Responsibilities Evaluate AI-generated responses to ensure accuracy...Remote jobContract workSummer work$85 per hour
...Mercor connects elite creative and technical talent with leading AI research labs. Headquartered in San Francisco, our investors... ...Responsibilities Use frontier AI coding agents to complete and evaluate complex infrastructure engineering tasks. Review model-...Remote jobContract workSummer work- ...Waabi, founded by AI visionary Raquel Urtasun, is the leader in Physical AI. With a world... ...positive way. To learn more visit: The Evaluation Algorithms team is responsible for... ...Proficient in Python programming and strong software engineering fundamentals with real-world experience...SoftwareFull timeInternship
$140k - $175k per year
...Lead Software Engineer, AI Are you passionate about building AI-driven software that transforms... ...systems that power the next generation of expert solutions across legal, tax, risk, and... ...& Technical Strategy ~ Lead evaluation and strategic adoption of cutting-edge...SoftwareFull timeWork at officeLocal areaFlexible hours2 days per week3 days per week$80 - $120 per hour
...Mercor connects elite creative and technical talent with leading AI research labs. Headquartered in San Francisco, our investors... ...Position: Compliance / regulatory response with financial-services AI Evaluator Type: Contract Compensation: $80–$120/hour...Remote jobContract workSummer workWork at office$85 per hour
...Mercor connects elite creative and technical talent with leading AI research labs. Headquartered in San Francisco, our investors... ...Responsibilities Use frontier AI coding agents to complete and evaluate complex engineering tasks. Review model-generated mobile...Remote jobContract workSummer work- ...combining professional expertise with Adaptive AI technology that is continuously refined by... ...LLM deployment. Work closely with our expert advisor to strategize, plan, and design... ...Work closely with DevOps Engineers, software engineers, designers, and product managers...SoftwareFull timeShift work
$80k - $100k per year
...Software Engineer - AI – CoCounsel Forward Deployed Engineering Are you excited about building AI... ...delivery, prototyping, integrations, and evaluation of AI outputs for legal use cases.... ...Human Resources Department at HR.Leave-Expert@thomsonreuters.com . Disability accommodations...SoftwareFull timeContract workInternshipWork at officeLocal areaFlexible hours2 days per week3 days per week$70 - $100 per hour
...Mercor connects elite creative and technical talent with leading AI research labs. Headquartered in San Francisco, our investors... ...Jack Dorsey . Position: STEM Computational Scientific Software & Evaluation Design - Astrophysics & Cosmology Type: Contract...SoftwareRemote jobContract workSummer work- ...no prior tech background — through cohort-based programs built on our own platform and curriculum, developed in partnership with Nebius AI. Our team is fully remote and globally distributed, and we serve a large, active student base across both regions. We're building a program...Work at officeRemote workFlexible hours
- ...systems, and facilities. The ideal candidate will demonstrate strong technical skillsincluding proficiency in HRIS systems and related software, and a solid understanding of confidentiality, compliance, and collaborative work environments. Duties ~ Coordinate recruitment...SoftwareFull timeImmediate startMonday to fridayShift work
$122k - $192k per year
...Resource Innovations is looking for an experienced DSM Evaluation Manager to join our expanding team. In this role, you will provide technical... ...with impact evaluation of energy efficiency programs. ~ Expert level understanding of building systems, energy end uses and equipment...Full timeInternshipWork at officeLocal area$20 - $22 per hour
...Mercor connects elite creative and technical talent with leading AI research labs. Headquartered in San Francisco, our investors include... ..., Larry Summers , and Jack Dorsey . Position: AI Safety Experts — English & Telugu Type: Contract Compensation:...Remote jobHourly payWeekly payContract workFor contractorsSummer workFlexible hours$55 - $80 per hour
...creative and technical talent with leading AI research labs. Headquartered in San... ...Position: Automated CS Planning PhDs & Domain Experts — Zion (Arches) Type: Contract... ...creating complex problems in your domain. Evaluate and improve AI model performance through...Remote jobContract workSummer work$20 - $22 per hour
...Mercor connects elite creative and technical talent with leading AI research labs. Headquartered in San Francisco, our investors include... ..., Larry Summers , and Jack Dorsey . Position: AI Safety Experts — English & Marathi Type: Contract Compensation:...Remote jobContract workSummer work$80 per hour
...Mercor connects elite creative and technical talent with leading AI research labs. Headquartered in San Francisco, our investors... ...Responsibilities Use frontier AI coding agents to complete and evaluate complex data engineering tasks. Review model-generated...Remote jobContract workSummer work$140k - $175k per year
...spanning Engagement Manager and Guided Assurance. As a Lead Software Engineer AI (Staff Engineer AI), you will own core backend and AI orchestration... ...AI integration practices across the org. ~ Be a go-to expert in new model capabilities, collaborating closely with AI/ML...SoftwareLong term contractFull timeWork at officeLocal areaFlexible hoursShift work2 days per week3 days per week$70 - $90 per hour
...connects elite creative and technical talent with leading AI research labs. Headquartered in San Francisco, our investors... ...Capture high-quality screenshots of professional CAD software during realistic, expert workflows. Annotate interactive UI elements such as buttons...SoftwareRemote jobContract workSummer work
Do you want to receive more vacancies?
Subscribe and receive similar vacancies to AI Evaluator - Software Expert. Be the first to apply!
Related searches
- estimateur en bâtiment Toronto, ON
- évaluateur-estimateur Toronto, ON
- estimateur structure Toronto, ON
- chargé de projet estimateur Toronto, ON
- admissions evaluator Toronto, ON
- remote social media evaluator Toronto, ON
- test evaluator Toronto, ON
- estimateur automobile Toronto, ON
- expert batiment Toronto, ON
- expert automobile Toronto, ON