AI Evaluator - Software Expert
$80 - $120 per hourMercor
Toronto, ON
- Remote job
About the job
Mercor connects elite creative and technical talent with leading AI research labs. Headquartered in San Francisco, our investors include Benchmark , General Catalyst , Peter Thiel , Adam D'Angelo , Larry Summers , and Jack Dorsey .
Position: Software / AI / IT / data Evaluator
Type: Contract
Compensation: $80–$120/hour
Location: Remote
Role Responsibilities
- Evaluate AI-generated artifacts against domain-specific quality rubrics.
- Identify factual, aesthetic, and presentation errors in documents, spreadsheets, and slide decks .
- Provide clear, structured written feedback to improve AI model outputs .
- Apply deep subject-matter expertise to grade outputs for accuracy and rigor.
- Work independently and asynchronously to meet deadlines.
Qualifications
Must-Have
- 5+ years of relevant professional experience in Software / AI / IT / data .
- Native or professional fluency in English .
- Highly proficient in Microsoft Office and Google Workspace , especially Slides .
Preferred
- Advanced degree ( Master's or higher ) from a reputable institution.
Application Process (Takes 20–30 mins to complete)
- Upload resume
- AI interview based on your resume
- Submit form
Resources & Support
- For details about the interview process and platform information, please check:
- For any help or support, reach out to: Show email
PS: Our team reviews applications daily. Please complete your AI interview and application steps to be considered for this opportunity.
Vacancy posted 1 day ago
Similar jobs that could be interesting for youBased on the AI Evaluator - Software Expert in Toronto, ON vacancy
$120 - $170 per hour
...creative and technical talent with leading AI research labs. Headquartered in San... ...Jack Dorsey . Position: Data Science Expert Type: Contract Compensation... ...~ Prior experience with AI training , evaluation, or human-data projects. Application...SuggestedRemote jobContract workSummer work$100 - $150 per hour
...technical talent with leading AI research labs. Headquartered in... ...Position: Sales Engineering Expert Type: Contract... ...supporting proofs of concept, and evaluating product fit and technical risk... ...with technically complex B2B software — SaaS , cloud infrastructure...SoftwareRemote jobContract workSummer work$80 - $120 per hour
...Mercor connects elite creative and technical talent with leading AI research labs. Headquartered in San Francisco, our investors... ...Jack Dorsey . Position: Privacy / regulatory compliance Evaluator Type: Contract Compensation: $80–$120/hour...SuggestedRemote jobContract workSummer workWork at officeFlexible hours$80 - $120 per hour
...Mercor connects elite creative and technical talent with leading AI research labs. Headquartered in San Francisco, our investors... ...Dorsey . Position: Legal contracts / diligence / redlines Evaluator Type: Contract Compensation: $80–$120/hour...SuggestedRemote jobContract workSummer workWork at office$80 - $120 per hour
...Mercor connects elite creative and technical talent with leading AI research labs. Headquartered in San Francisco, our investors... ...Summers , and Jack Dorsey . Position: Cybersecurity / IT GRC Evaluator Type: Contract Compensation: $80–$120/hour...SuggestedRemote jobContract workSummer workWork at office$80 - $120 per hour
...creative and technical talent with leading AI research labs. Headquartered in San... .... Position: Healthcare / clinical Evaluator Type: Contract Compensation... ...and asynchronously to ensure high-quality evaluations. Utilize Microsoft Office and Google...Remote jobContract workSummer workWork at office$100 - $150 per hour
...creative and technical talent with leading AI research labs. Headquartered in San... ...Jack Dorsey . Position: B2B Sales Expert Type: Contract Compensation... ...~ Prior experience with AI training , evaluation, or human-data projects. Application...Remote jobContract workSummer work$14 per hour
...creative and technical talent with leading AI research labs. Headquartered in San... ...Jack Dorsey . Position: Music Audio Expert - Telugu Type: Contract Compensation... .../week Role Responsibilities Evaluate AI model output lyrics , voice...Remote jobContract workSummer workImmediate start$80 - $150 per hour
...technical talent with leading AI research labs. Headquartered in... ...Position: UI / UX Design Expert Type: Contract... ...independently and asynchronously to evaluate and improve AI model performance... ...with AI training , evaluation, or human-data projects....Remote jobContract workSummer work$80k - $180k per year
...Zafin is an AI platform company helping regulated institutions modernize how critical... ...scale. What’s the Opportunity? The AI Evaluation Engineer ensures AI agent solutions are accurate... ...3–10+ years of relevant experience in software quality engineering, AI evaluation, AI...SoftwareFull time$100 - $150 per hour
...creative and technical talent with leading AI research labs. Headquartered in San... ...pipelines , and A/B test write-ups . Evaluate AI-generated or human-created work against... ...detailed written justifications for evaluations and scores to maintain transparency and accountability...Remote jobContract workSummer work$60 - $75 per hour
...creative and technical talent with leading AI research labs. Headquartered in San... ..., and Jack Dorsey . Position: Software & Data Science Expert Type: Contract... ...reasoning over technical documentation. Evaluate AI ability to follow precise instructions...SoftwareRemote jobContract workSummer work$85 per hour
...Mercor connects elite creative and technical talent with leading AI research labs. Headquartered in San Francisco, our investors... ...Responsibilities Use frontier AI coding agents to complete and evaluate complex infrastructure engineering tasks. Review model-...Remote jobContract workSummer work$80 per hour
...Mercor connects elite creative and technical talent with leading AI research labs. Headquartered in San Francisco, our investors... ...Responsibilities Use frontier AI coding agents to complete and evaluate complex data engineering tasks. Review model-generated...Remote jobContract workSummer work$80 - $120 per hour
...Mercor connects elite creative and technical talent with leading AI research labs. Headquartered in San Francisco, our investors... ...Jack Dorsey . Position: Biology / environmental science Evaluator Type: Contract Compensation: $80–$120/hour...Remote jobContract workSummer workWork at office$149k - $168k per year
...dealerships nationwide, delivering cutting-edge software tools to simplify operations, boost... ...We are looking for a hands-on Engineering Expert who will be: setting technical direction across... ...of progression. Help teams adopt AI-assisted engineering safely across coding,...SoftwareInternship$80 - $120 per hour
...creative and technical talent with leading AI research labs. Headquartered in San... ...Position: Training / onboarding / L&D Evaluator Type: Contract Compensation... ...feedback. Collaborate with subject matter experts to ensure consistency and quality. Work...Remote jobContract workSummer workWork at office$140k - $175k per year
...Staff Software Engineer, AI Ready to build the AI that transforms how professionals work? Join a... ...systems powering the next generation of expert solutions across legal, tax, risk, and... ...Technical Strategy Lead the charge in evaluating and adopting cutting-edge AI techniques...SoftwareFull timeInternshipWork at officeLocal areaFlexible hours$140k - $175k per year
...Lead Software Engineer, AI What if the AI agents you architect empower legal professionals to... ...services. Design agent observability and evaluation practices (tracing, automated evals,... ...Human Resources Department at HR.Leave-Expert@thomsonreuters.com . Disability...SoftwareFull timeInternshipWork at officeLocal areaFlexible hoursShift work$85 per hour
...Mercor connects elite creative and technical talent with leading AI research labs. Headquartered in San Francisco, our investors... ...Responsibilities Use frontier AI coding agents to complete and evaluate complex machine learning and AI engineering tasks....Remote jobContract workSummer work- ...curriculum, developed in partnership with Nebius AI. Our team is fully remote and globally... ...of expertise We will bring on three Experts , each focused on one of the following... ...you REQUIREMENTS ~5+ years as a senior software/systems architect, platform engineer, or senior...SoftwareWork at officeRemote workFlexible hours
$29 - $45 per hour
...Mercor connects elite creative and technical talent with leading AI research labs. Headquartered in San Francisco, our investors include... ..., Larry Summers , and Jack Dorsey . Position: AI Safety Experts — English & Portuguese (global) Type: Contract...Remote jobContract workSummer work$48 - $62 per hour
...Mercor connects elite creative and technical talent with leading AI research labs. Headquartered in San Francisco, our investors include... ..., Larry Summers , and Jack Dorsey . Position: AI Safety Experts — English & Dutch Type: Contract Compensation:...Remote jobContract workSummer work$68k - $72k per year
...team is looking to fill a new position of Manager, Planning and Evaluation. This is a full-time position working 35 hours per week anywhere... ...relationships with the land. As part of our recruitment process, we use AI-enabled tools to assist with summarizing applicant information...Full timeInternshipWork at officeFlexible hoursWeekend work$85 per hour
...Mercor connects elite creative and technical talent with leading AI research labs. Headquartered in San Francisco, our investors... ...Responsibilities Use frontier AI coding agents to complete and evaluate complex engineering tasks. Review model-generated mobile...Remote jobContract workSummer work- ...Period: 06-Aug-2026 to 20-Aug-2026 Reporting to the Manager, Evaluation and Quality Improvement, the Program Evaluator works collaboratively... ...of program staff, management, senior management and technical experts. ~Credentialed Evaluator designation, is an asset....Hourly payFull timeTemporary workPart timeInternshipMonday to friday
- ...no prior tech background — through cohort-based programs built on our own platform and curriculum, developed in partnership with Nebius AI. Our team is fully remote and globally distributed, and we serve a large, active student base across both regions. We're building a program...Work at officeRemote workFlexible hours
- ...combining professional expertise with Adaptive AI technology that is continuously refined by... ...LLM deployment. Work closely with our expert advisor to strategize, plan, and design... ...Work closely with DevOps Engineers, software engineers, designers, and product managers...SoftwareFull timeShift work
$145 per hour
...technical talent with leading AI research labs. Headquartered in... ...Position: ML Research PhD Experts (ICML / NeurIPS / ICLR Publications... ...Role Responsibilities Evaluate the accuracy and depth of AI-... ...performance. Develop evaluation rubrics and benchmarks for assessing...Remote jobContract workSummer work$85 per hour
...Mercor connects elite creative and technical talent with leading AI research labs. Headquartered in San Francisco, our investors... ...Responsibilities Use frontier AI coding agents to complete and evaluate complex engineering tasks. Review model-generated frontend...Remote jobContract workSummer work
Do you want to receive more vacancies?
Subscribe and receive similar vacancies to AI Evaluator - Software Expert. Be the first to apply!
Related searches
- remote social media evaluator Toronto, ON
- estimateur automobile Toronto, ON
- estimateur en bâtiment Toronto, ON
- admissions evaluator Toronto, ON
- évaluateur-estimateur Toronto, ON
- test evaluator Toronto, ON
- estimateur structure Toronto, ON
- chargé de projet estimateur Toronto, ON
- expert automobile Toronto, ON
- expert batiment Toronto, ON