AI Evaluator - Software Expert
$80 - $120 per hourMercor
Toronto, ON
- Remote job
About the job
Mercor connects elite creative and technical talent with leading AI research labs. Headquartered in San Francisco, our investors include Benchmark , General Catalyst , Peter Thiel , Adam D'Angelo , Larry Summers , and Jack Dorsey .
Position: Software / AI / IT / data Evaluator
Type: Contract
Compensation: $80–$120/hour
Location: Remote
Role Responsibilities
- Evaluate AI-generated artifacts against domain-specific quality rubrics.
- Identify factual, aesthetic, and presentation errors in documents, spreadsheets, and slide decks .
- Provide clear, structured written feedback to improve AI model outputs .
- Apply deep subject-matter expertise to grade outputs for accuracy and rigor.
- Work independently and asynchronously to meet deadlines.
Qualifications
Must-Have
- 5+ years of relevant professional experience in Software / AI / IT / data .
- Native or professional fluency in English .
- Highly proficient in Microsoft Office and Google Workspace , especially Slides .
Preferred
- Advanced degree (Master's or higher) from a reputable institution.
Application Process (Takes 20–30 mins to complete)
- Upload resume
- AI interview based on your resume
- Submit form
Resources & Support
- For details about the interview process and platform information, please check:
- For any help or support, reach out to: Show email
PS: Our team reviews applications daily. Please complete your AI interview and application steps to be considered for this opportunity.
Vacancy posted 1 day ago
Similar jobs that could be interesting for youBased on the AI Evaluator - Software Expert in Toronto, ON vacancy
$80 - $120 per hour
...Mercor connects elite creative and technical talent with leading AI research labs. Headquartered in San Francisco, our investors... ...Jack Dorsey . Position: Privacy / regulatory compliance Evaluator Type: Contract Compensation: $80–$120/hour...SuggestedRemote jobContract workSummer workWork at office$80 - $120 per hour
...Mercor connects elite creative and technical talent with leading AI research labs. Headquartered in San Francisco, our investors... ...Jack Dorsey . Position: Biology / environmental science Evaluator Type: Contract Compensation: $80–$120/hour...SuggestedRemote jobContract workSummer workWork at office$80 - $120 per hour
...Mercor connects elite creative and technical talent with leading AI research labs. Headquartered in San Francisco, our investors... ...and Jack Dorsey . Position: IP / trademark / copyright law Evaluator Type: Contract Compensation: $80–$120/hour...SuggestedRemote jobContract workSummer workWork at office$80 - $120 per hour
...creative and technical talent with leading AI research labs. Headquartered in San... ...Position: Pricing / ROI / revenue economics Evaluator Type: Contract Compensation... ...meet deadlines and ensure high-quality evaluations. Qualifications Must-Have...SuggestedRemote jobContract workSummer workWork at office$70 per hour
...creative and technical talent with leading AI research labs. Headquartered in San... ...Jack Dorsey . Position: Generalist Expert Type: Contract Compensation... ...Remote Role Responsibilities Evaluate AI-generated responses to ensure accuracy...SuggestedRemote jobContract workSummer work$80 - $120 per hour
...creative and technical talent with leading AI research labs. Headquartered in San... ...Operations / inventory / capacity planning Evaluator Type: Contract Compensation... ...meet deadlines and ensure high-quality evaluations. Qualifications Must-Have...Remote jobContract workSummer workWork at office$15 - $20 per hour
...Mercor connects elite creative and technical talent with leading AI research labs. Headquartered in San Francisco, our investors... ...sources and external tools . Generate high-quality human evaluation data by identifying response strengths, areas for improvement, and...Remote jobContract workSummer work$80k - $90k per year
...Senior Manager, Impact and Evaluation Existing vacancy. No AI used in screening or selection. Founded in 2011 by His Majesty King Charles III, The King’s Trust Canada (TKTC) is a national charity that helps young people build the skills, networks, and experiences they...Full timeFor contractorsCasual workWork at officeAfternoon shift3 days per week$80 - $120 per hour
...Mercor connects elite creative and technical talent with leading AI research labs. Headquartered in San Francisco, our investors... ...Position: Compliance / regulatory response with financial-services AI Evaluator Type: Contract Compensation: $80–$120/hour...Remote jobContract workSummer workWork at office$85 per hour
...Mercor connects elite creative and technical talent with leading AI research labs. Headquartered in San Francisco, our investors... ...Responsibilities Use frontier AI coding agents to complete and evaluate complex engineering tasks. Review model-generated mobile...Remote jobContract workSummer work- ...system could directly improve how AI advises the next generation of... ...and Physicians in Toronto to evaluate and improve AI-generated... ..."golden answers" that reflect expert-level career guidance within the... ...medical field Create objective evaluation rubrics used to assess AI-...Hourly payOngoing contractLong term contractContract workFreelanceInternshipRemote workFlexible hours
- ...excited to work with you REQUIREMENTS ~5+ years as a senior software/systems architect, platform engineer, or senior backend/infrastructure... ...C1+ — instruction and review are in English for a US-based audience ~ Comfortable using AI tools in day-to-day technical work...SoftwareWork at officeRemote workFlexible hours
- 247labs is a leading software development company headquartered in Toronto... ...looking for a Senior SEO & AI Engine Optimization Lead who... ...with how AI search platforms evaluate credibility and relevance Position... ...Act as the internal expert and advocate for organic and AI...SoftwareFull timePart timeRemote work
$70 - $100 per hour
...Mercor connects elite creative and technical talent with leading AI research labs. Headquartered in San Francisco, our investors... ...Jack Dorsey . Position: STEM Computational Scientific Software & Evaluation Design - Astrophysics & Cosmology Type: Contract...SoftwareRemote jobContract workSummer work$90k per year
...cloud platform, coupled with AI-driven analytics tools, unlocks... ...highly skilled and experienced Software Engineer, AI to perform a key... ...engineering, data pipelines, evaluation, and production operations. You... ...development from industry-leading experts ~ Cutting edge benefit...SoftwareFlexible hours- ...meaningful work-life balance. Being a Senior AI Software Engineer at iManage Means… You are... ...AI systems from model development and evaluation through to scalable production serving,... ..., and empowers parental success through expert-led training and resources. iManage...SoftwareFull timeWork at officeLocal areaWorldwideFlexible hours
$85 per hour
...Mercor connects elite creative and technical talent with leading AI research labs. Headquartered in San Francisco, our investors... ...Responsibilities Use frontier AI coding agents to complete and evaluate complex engineering tasks. Review model-generated frontend...Remote jobContract workSummer work- ...directly shape how the next generation of AI understands and generates software? We're looking for Software Engineers in Toronto to build, evaluate, and improve AI-generated code — helping... ..., review AI outputs, and provide the expert-level feedback that makes AI coding assistants...SoftwareHourly payOngoing contractContract workFreelanceManual laborRemote workFlexible hours
$80 per hour
...Mercor connects elite creative and technical talent with leading AI research labs. Headquartered in San Francisco, our investors... ...Responsibilities Use frontier AI coding agents to complete and evaluate complex data engineering tasks. Review model-generated...Remote jobContract workSummer work- ...meaningful work-life balance. Being a Staff AI Software Engineer at iManage Means… You are... ...production, from model development and evaluation to scalable application design and... ..., and empowers parental success through expert-led training and resources. iManage is...SoftwareFull timeWork at officeLocal areaWorldwideFlexible hours
- ...professional expertise with Adaptive AI technology that is... ...prototyping, and collaborate with the software team to deploy models. We... ...hardware platforms. Train and evaluate generative models using... ...Huggingface APIs or Amazon Bedrock. ~ Expert skills of Python, including...SoftwareFull timeInternshipWork at officeRemote workShift work
$140k - $175k per year
...Lead Software Engineer, AI Are you passionate about building AI-driven software that transforms... ...systems that power the next generation of expert solutions across legal, tax, risk, and... ...& Technical Strategy ~ Lead evaluation and strategic adoption of cutting-edge...SoftwareFull timeWork at officeLocal areaFlexible hours2 days per week3 days per week$70 - $90 per hour
...connects elite creative and technical talent with leading AI research labs. Headquartered in San Francisco, our investors... ...Capture high-quality screenshots of professional CAD software during realistic, expert workflows. Annotate interactive UI elements such as buttons...SoftwareRemote jobContract workSummer work$20 - $22 per hour
...Mercor connects elite creative and technical talent with leading AI research labs. Headquartered in San Francisco, our investors include... ..., Larry Summers , and Jack Dorsey . Position: AI Safety Experts — English & Urdu Type: Contract Compensation: $...Remote jobContract workSummer work$20 - $22 per hour
...Mercor connects elite creative and technical talent with leading AI research labs. Headquartered in San Francisco, our investors include... ..., Larry Summers , and Jack Dorsey . Position: AI Safety Experts — English & Punjabi Type: Contract Compensation:...Remote jobContract workSummer work$20 - $22 per hour
...Mercor connects elite creative and technical talent with leading AI research labs. Headquartered in San Francisco, our investors include... ..., Larry Summers , and Jack Dorsey . Position: AI Safety Experts — English & Kannada Type: Contract Compensation:...Remote jobContract workSummer work$100 per hour
...Mercor connects elite creative and technical talent with leading AI research labs. Headquartered in San Francisco, our investors... ...leading foundational model AI labs to improve AI model training and evaluation. Work independently and asynchronously to meet deadlines...SoftwareRemote jobHourly payContract workFor contractorsSummer work- ...Staff Software Engineer (AI) – Forward Engineering Build the future of AI‑powered systems at Thomson... ...by leading proofs of concept, evaluating emerging techniques, and translating research... ...Human Resources Department at HR.Leave-Expert@thomsonreuters.com . Disability...SoftwareFull timeWork at officeLocal areaFlexible hours2 days per week3 days per week
- ...Senior AI Application Engineer Location: Woodbridge, ON (Hybrid - 3 days onsite per... ...non-deterministic AI behavior. • Develop evaluation frameworks and testing strategies for AI systems... ...harnesses. • Strong understanding of software engineering best practices, SDLC...SoftwareContract workWork at office3 days per week
- ...Senior Conversation Designer – AI Experiences Location: Toronto, ON (Hybrid) Contract... ...Participate in usability testing and evaluate AI response quality to continuously improve... ...Experience defining content quality metrics or evaluation frameworks Knowledge of accessibility...Hourly payContract work
Do you want to receive more vacancies?
Subscribe and receive similar vacancies to AI Evaluator - Software Expert. Be the first to apply!
Related searches
- admissions evaluator Toronto, ON
- évaluateur-estimateur Toronto, ON
- chargé de projet estimateur Toronto, ON
- estimateur en bâtiment Toronto, ON
- test evaluator Toronto, ON
- estimateur automobile Toronto, ON
- web search evaluator Toronto, ON
- estimateur structure Toronto, ON
- remote social media evaluator Toronto, ON
- expert batiment Toronto, ON