AI Evaluator - Software Expert
$80 - $120 per hourMercor
Toronto, ON
- Remote job
About the job
Mercor connects elite creative and technical talent with leading AI research labs. Headquartered in San Francisco, our investors include Benchmark , General Catalyst , Peter Thiel , Adam D'Angelo , Larry Summers , and Jack Dorsey .
Position: Software / AI / IT / data Evaluator
Type: Contract
Compensation: $80–$120/hour
Location: Remote
Role Responsibilities
- Evaluate AI-generated artifacts against domain-specific quality rubrics.
- Identify factual, aesthetic, and presentation errors in documents, spreadsheets, and slide decks .
- Provide clear, structured written feedback to improve AI model outputs .
- Apply deep subject-matter expertise to grade outputs for accuracy and rigor.
- Work independently and asynchronously to meet deadlines.
Qualifications
Must-Have
- 5+ years of relevant professional experience in Software / AI / IT / data .
- Native or professional fluency in English .
- Highly proficient in Microsoft Office and Google Workspace , especially Slides .
Preferred
- Advanced degree ( Master's or higher ) from a reputable institution.
Application Process (Takes 20–30 mins to complete)
- Upload resume
- AI interview based on your resume
- Submit form
Resources & Support
- For details about the interview process and platform information, please check:
- For any help or support, reach out to: Show email
PS: Our team reviews applications daily. Please complete your AI interview and application steps to be considered for this opportunity.
Vacancy posted a month ago
Similar jobs that could be interesting for youBased on the AI Evaluator - Software Expert in Toronto, ON vacancy
$80 - $120 per hour
...connects elite creative and technical talent with leading AI research labs. Headquartered in San Francisco, our investors... ...Angelo , Larry Summers , and Jack Dorsey . Position: Software / AI / IT / data Evaluator Type: Contract Compensation: $80–$120/...SoftwareRemote jobContract workSummer workWork at office$100 - $150 per hour
...creative and technical talent with leading AI research labs. Headquartered in San... ..., and Jack Dorsey . Position: Software Engineer Expert Type: Contract Compensation... ...Role Responsibilities Evaluate AI-generated artifacts for usability...SoftwareContract workSummer workWork at officeRemote work$70 per hour
...creative and technical talent with leading AI research labs. Headquartered in San... .... Position: Mathematics PhD Coding Experts Type: Contract Compensation... ...-reviewed journals. Prior scientific software or research engineering experience. Start...SoftwareRemote jobContract workSummer workImmediate start$120 - $170 per hour
...creative and technical talent with leading AI research labs. Headquartered in San... ...Jack Dorsey . Position: Data Science Expert Type: Contract Compensation... ...~ Prior experience with AI training , evaluation, or human-data projects. Application...SuggestedFull timeContract workSummer workRemote work$100 per hour
...candidates based in Canada. Pay: $100$200/hr Role Title: AI Software Engineering Domain Expert Role Type: Contractor (Part Time) Location: Remote... ...practices in software engineering. # Create refine and evaluate prompts and instructions designed to guide AI models in...SoftwareFull timePart timeFor contractorsRemote work$100 - $150 per hour
...technical talent with leading AI research labs. Headquartered in... ...Position: Sales Engineering Expert Type: Contract... ...supporting proofs of concept, and evaluating product fit and technical risk... ...with technically complex B2B software — SaaS , cloud infrastructure...SoftwareFull timeContract workSummer workRemote work$100 - $150 per hour
...creative and technical talent with leading AI research labs. Headquartered in San... ...Jack Dorsey . Position: Data Science Expert Type: Contract Compensation... ...Remote Role Responsibilities Evaluate AI-generated artifacts for usability in...Remote jobContract workSummer workWork at office$100 - $150 per hour
...creative and technical talent with leading AI research labs. Headquartered in San... .... Position: Management Consulting Expert Type: Contract Compensation... ...Remote Role Responsibilities Evaluate AI-generated artifacts for usability in a...Full timeContract workSummer workWork at officeRemote work$50 per hour
...technical talent with leading AI research labs. Headquartered in... ...Language and Literature Expert Type: Contract... ...Role Responsibilities Evaluate the quality and accuracy of LLM... ...linguistic annotation, language evaluation, or NLP data work in English...Remote jobContract workSummer work$80 - $120 per hour
...Mercor connects elite creative and technical talent with leading AI research labs. Headquartered in San Francisco, our investors... ...Jack Dorsey . Position: Privacy / regulatory compliance Evaluator Type: Contract Compensation: $80–$120/hour...Full timeContract workSummer workWork at officeRemote work$20 - $160 per hour
...creative and technical talent with leading AI research labs. Headquartered in San... ...Annotator — Health AI Conversation Quality Evaluation Type: Contract Compensation... ...question in understandable language for non-experts. Detect sycophancy in AI interactions...Remote jobContract workSummer work$80 - $120 per hour
...Mercor connects elite creative and technical talent with leading AI research labs. Headquartered in San Francisco, our investors... ...Dorsey . Position: Public-sector procurement / RFI response Evaluator Type: Contract Compensation: $80–$120/hour...Full timeContract workSummer workWork at officeRemote workFlexible hours- ...service creation. The technology department is focused on delivering software solutions that enable rich consumer experiences, from mobile... ...digital experiences and client partnerships. As a QA / AI Evaluation Engineer, you will join a highly motivated and experienced team...SoftwareFull timeImmediate start
$80 - $120 per hour
...Mercor connects elite creative and technical talent with leading AI research labs. Headquartered in San Francisco, our investors... ...Summers , and Jack Dorsey . Position: Legal / compliance Evaluator Type: Contract Compensation: $80–$120/hour...Full timeContract workSummer workWork at officeRemote work$60 - $80 per hour
...connects elite creative and technical talent with leading AI research labs. Headquartered in San Francisco, our investors... ...Summers , and Jack Dorsey . Position: Business Domain Expert — AI Model Evaluation Pilot (Admin, Marketing, HR, Accounting) Type:...Hourly payFull timeContract workFor contractorsSummer workRemote work$80 - $120 per hour
...creative and technical talent with leading AI research labs. Headquartered in San... ...Position: Humanities / arts / culture Evaluator Type: Contract Compensation... .... Collaborate with subject matter experts to ensure consistency and quality. Work...Remote jobContract workSummer workWork at office$80 - $120 per hour
...Mercor connects elite creative and technical talent with leading AI research labs. Headquartered in San Francisco, our investors... ...Dorsey . Position: Public-sector procurement / RFI response Evaluator Type: Contract Compensation: $80–$120/hour...Remote jobContract workSummer workWork at office$120 - $170 per hour
...creative and technical talent with leading AI research labs. Headquartered in San... ...Jack Dorsey . Position: Data Science Expert Type: Contract Compensation... ...~ Prior experience with AI training , evaluation, or human-data projects. Application...Full timeContract workSummer workRemote work$80 - $120 per hour
...Mercor connects elite creative and technical talent with leading AI research labs. Headquartered in San Francisco, our investors... ...Position: Compliance / regulatory response with financial-services AI Evaluator Type: Contract Compensation: $80–$120/hour...Full timeContract workSummer workWork at officeRemote work$125k - $175k per year
...seeking to recruit the best and the brightest talents. The Senior AI Software Developer will design, build, and deliver the firm’s AI... ...developers and conducting formal code reviews. Qualifications Expert proficiency in Python, including asynchronous development,...SoftwareFull timeWork at office$100 - $150 per hour
...creative and technical talent with leading AI research labs. Headquartered in San... ...pipelines , and A/B test write-ups . Evaluate AI-generated or human-created work against... ...detailed written justifications for evaluations and scores to maintain transparency and accountability...Full timeContract workSummer workRemote work- ...About the Role SJC Media is building the AI layer for its Digital Operations. As the founding Software Engineer, Applied AI, you own that layer end-to-end... ...to working system in production. Architect well: Evaluate and select the right tools, LLMs, and platforms (Claude...SoftwareFull timeFlexible hours
$140 per hour
...Canada. Pay: $140$200/hr Role Title: AI Domain Expert Role Type: Contractor (Part Time)... ...AI model responses and performance. # Evaluate and annotate data with keen attention to... ...#3 years of experience in fields such as Software Engineering Finance Data Science or...SoftwareFull timePart timeFor contractorsRemote work$50 - $70 per hour
...creative and technical talent with leading AI research labs. Headquartered in San... ...Jack Dorsey . Position: Generalist Expert (US/Canada) Type: Contract... ...Role Responsibilities Review and evaluate documents, slides, spreadsheets, and similar...Remote jobContract workSummer workFlexible hours$100 - $150 per hour
...creative and technical talent with leading AI research labs. Headquartered in San... ...pipelines , and technical reports. Evaluate AI-generated or human-created work against... ...Provide detailed written justifications for evaluations and scores to maintain transparency....Remote jobContract workSummer work$140k - $175k per year
...with speed and certainty. As a Staff Software Engineer – AI on the Confirmation Product... ...workflows, safety controls, guardrails, evaluation, and resilient error handling around third... ...Human Resources Department at HR.Leave-Expert@thomsonreuters.com . Disability accommodations...SoftwareLong term contractFull timeWork at officeLocal areaFlexible hours$90k per year
...cloud platform, coupled with AI-driven analytics tools, unlocks... ...highly skilled and experienced Software Engineer, AI to perform a key... ...engineering, data pipelines, evaluation, and production operations. You... ...development from industry-leading experts ~ Cutting edge benefit...SoftwareFull timeFlexible hours$70 per hour
...creative and technical talent with leading AI research labs. Headquartered in San... ...Jack Dorsey . Position: Generalist Expert Type: Contract Compensation... ...Remote Role Responsibilities Evaluate AI-generated responses to strengthen reasoning...Remote jobContract workSummer work$80 - $120 per hour
...Mercor connects elite creative and technical talent with leading AI research labs. Headquartered in San Francisco our investors... ...and Jack Dorsey . Position: Process improvement / SOPs Evaluator Type: Contract Compensation: $80$120/hour...Full timeContract workSummer workWork at officeRemote work$80 per hour
...Mercor connects elite creative and technical talent with leading AI research labs. Headquartered in San Francisco, our investors... ...Responsibilities Use frontier AI coding agents to complete and evaluate complex data engineering tasks. Review model-generated...Full timeContract workSummer workRemote work
Do you want to receive more vacancies?
Subscribe and receive similar vacancies to AI Evaluator - Software Expert. Be the first to apply!
Related searches
- test evaluator Toronto, ON
- estimateur en bâtiment Toronto, ON
- estimateur structure Toronto, ON
- estimateur automobile Toronto, ON
- admissions evaluator Toronto, ON
- estimateur - dessinateur Toronto, ON
- chargé de projet estimateur Toronto, ON
- remote social media evaluator Toronto, ON
- estimateur industriel Toronto, ON
- évaluateur-estimateur Toronto, ON