Operations Evaluator - Domain Expert
$80 - $120 per hourMercor
Toronto, ON
- Remote job
About the job
Mercor connects elite creative and technical talent with leading AI research labs. Headquartered in San Francisco, our investors include Benchmark , General Catalyst , Peter Thiel , Adam D'Angelo , Larry Summers , and Jack Dorsey .
Position: Operations / inventory / capacity planning Evaluator
Type: Contract
Compensation: $80–$120/hour
Location: Remote
Role Responsibilities
- Evaluate AI-generated artifacts against domain-specific quality rubrics.
- Identify factual, aesthetic, and presentation errors in documents, spreadsheets, and slide decks.
- Provide clear, structured written feedback to improve AI model outputs .
- Apply deep subject-matter expertise in Operations , inventory , and capacity planning .
- Work independently and asynchronously to meet deadlines and ensure high-quality evaluations.
Qualifications
Must-Have
- 5+ years of relevant professional experience in Operations , inventory , and capacity planning .
- Native or professional fluency in English .
- Highly proficient in Microsoft Office and Google Workspace , especially Slides (Google Slides / PowerPoint).
Preferred
- Advanced degree (Master's or higher) from a reputable institution.
Application Process (Takes 20–30 mins to complete)
- Upload resume
- AI interview based on your resume
- Submit form
Resources & Support
- For details about the interview process and platform information, please check:
- For any help or support, reach out to: Show email
PS: Our team reviews applications daily. Please complete your AI interview and application steps to be considered for this opportunity.
Vacancy posted 29 days ago
Similar jobs that could be interesting for youBased on the Operations Evaluator - Domain Expert in Toronto, ON vacancy
$80 - $120 per hour
...and Jack Dorsey . Position: General finance / accounting Evaluator Type: Contract Compensation: $80–$120/hour... ...Responsibilities Evaluate AI-generated artifacts against domain-specific quality rubrics. Identify factual, aesthetic, and...SuggestedRemote jobContract workSummer workWork at office$80 - $120 per hour
...Jack Dorsey . Position: Biology / environmental science Evaluator Type: Contract Compensation: $80–$120/hour... ...Responsibilities Evaluate AI-generated artifacts against domain-specific quality rubrics. Identify factual, aesthetic, and...SuggestedRemote jobContract workSummer workWork at office$80 - $120 per hour
...Summers , and Jack Dorsey . Position: Education / school Evaluator Type: Contract Compensation: $80–$120/hour... ...Responsibilities Evaluate AI-generated artifacts against domain-specific quality rubrics. Identify factual, aesthetic, and...SuggestedRemote jobContract workSummer workWork at office$80 - $120 per hour
...Larry Summers , and Jack Dorsey . Position: Healthcare operations Evaluator Type: Contract Compensation: $80–$120/hour... ...Evaluate AI-generated artifacts against domain-specific quality rubrics. Identify factual, aesthetic, and...SuggestedRemote jobContract workSummer workWork at office$80 - $120 per hour
...Position: Training / onboarding / L&D Evaluator Type: Contract Compensation... ...Evaluate AI-generated artifacts against domain-specific quality rubrics. Identify factual... .... Collaborate with subject matter experts to ensure consistency and quality....SuggestedRemote jobContract workSummer workWork at office$80 - $120 per hour
...Summers , and Jack Dorsey . Position: Cybersecurity / IT GRC Evaluator Type: Contract Compensation: $80–$120/hour... ...Evaluate AI-generated artifacts against domain-specific quality rubrics. Identify factual, aesthetic, and...Remote jobContract workSummer workWork at office$80 - $120 per hour
...Position: Humanities / arts / culture Evaluator Type: Contract Compensation... ...AI-generated artifacts against domain-specific quality rubrics. Identify factual... ...to meet deadlines and ensure high-quality evaluations. Qualifications Must-Have...Remote jobContract workSummer workWork at office$80 - $120 per hour
...Summers , and Jack Dorsey . Position: Healthcare / clinical Evaluator Type: Contract Compensation: $80–$120/hour... ...Evaluate AI-generated artifacts against domain-specific quality rubrics. Identify factual, aesthetic, and...Remote jobContract workSummer workWork at office$15 - $20 per hour
...sources and external tools . Generate high-quality human evaluation data by identifying response strengths, areas for improvement, and... ...Strong attention to detail . Background or experience in domains requiring structured analytical thinking . Preferred...Remote jobContract workSummer work$80 per hour
..., and Jack Dorsey . Position: STEM PhDs and Technical Domain Experts Type: Contract Compensation: $55–$80/hour... ...innovation by developing challenging problems in your domain. Evaluate and improve AI model training and outputs to ensure accuracy...Remote jobContract workSummer work- ...Create deliverables for common requests within your professional domain. Review peer-developed deliverables to improve AI research... ...for around 3–4 weeks . Compensation & Legal ~ All experts receive a $100 onboarding payment for completing the onboarding...Remote jobHourly payContract workSummer workImmediate startTrial periodShift work
- ...want to impact the world in a positive way. To learn more visit: We are looking for a hands-on leader to build a new centralized Evaluation team. This team will be responsible for providing comprehensive and holistic analysis on all aspects of performance of the autonomy...Full time
- ...QA Lead - Insurance Domain Location: Toronto, ON Work Model: Hybrid – 3 days at client office Experience Required: 10+ years Skills ~ EIS: QA Inspection and Testing Skill Role Description Define and own end-to-end test strategy across...Contract workWork at office
$50 per hour
...and Jack Dorsey . Position: Internet-Native Bilingual Evaluator Expert (French) Type: Contract Compensation: $50/... ...cultural references, sarcasm, and code-switching. Build evaluation rubrics for bilingual internet communication and define expectations...Remote jobContract workSummer work10 hours per week- Job Title: Machine Operator 1 FLSA Status: Non-Exempt Department: Operations Reports to: Assistant Manager, Superintendent or Assistant Superintendent... ...Exam, Regulated (DOT) DS/BAT, Human Performance Evaluation (HPE), and Audiogram. - Position requires successful...Remote workShift work
- ...We are seeking Audio Evaluators to participate in an exciting project focused on evaluating audio clips in English (Pakistani) . This... ...• Apply consistent and objective judgment based on provided evaluation guidelines. • Maintain high quality and timely completion of...Remote jobContract workFreelance
- ...Role: AWS Solution Architect (Payment Domain) Total Experience: 6–8 Years Required Skill Set # CI/CD & Automation: Design,... ...Key Vault. # Scripting & Optimization: Write automation scripts using Ansible, PowerShell, Bash, or Python to reduce operational toil....Contract work
$70 per hour
...Angelo , Larry Summers , and Jack Dorsey . Position: Generalist Expert Type: Contract Compensation: $70/hour Location: Remote Role Responsibilities Evaluate AI-generated responses to ensure accuracy and depth in reasoning....Remote jobContract workSummer work$80 - $120 per hour
...Position: Compliance / regulatory response with financial-services AI Evaluator Type: Contract Compensation: $80–$120/hour... ...Evaluate AI-generated artifacts against domain-specific quality rubrics. Identify factual, aesthetic, and...Remote jobContract workSummer workWork at office- ...looking for diverse, innovative and collaborative candidates who want to impact the world in a positive way. To learn more visit: The Evaluation Algorithms team is responsible for building the algorithms & tooling required to comprehensively evaluate our autonomy system’s...Full timeInternship
$85 per hour
...Compensation: $85/hour Location: Remote Role Responsibilities Use frontier AI coding agents to complete and evaluate complex infrastructure engineering tasks. Review model-generated implementations involving cloud platforms , Kubernetes ,...Remote jobContract workSummer work- ...Lead Java Developer - Payment Domain (Banking) Location: Toronto, ON – Hybrid (4 Days WFO) Need Lead Developer profiles with the below-mentioned skillsets and payment domain experience. Candidate should be available for an in-person interview. Candidate should...Contract work
$85 per hour
...Compensation: $85/hour Location: Remote Role Responsibilities Use frontier AI coding agents to complete and evaluate complex engineering tasks. Review model-generated mobile application code for correctness, quality, maintainability, and...Remote jobContract workSummer work$142k - $192k per year
...Services (ECS) Delivery organization is to run and operate SAP’s private cloud offering. The Enhanced Operation Service is part of SAP Enterprise Cloud Services Delivery... ...Database Technologies like HANA/ASE/MSSQL – with expert level knowledge in HANA • You demonstrate the...Long term contractPermanent employmentFull timeLocal areaWorldwideFlexible hours$72k - $125k per year
...largest 100% Canadian-owned and operated professional services firm in... ...from deep subject matter experts through mentoring and on the job... ...feedback loops ML Flow for model evaluation and drift detection... ...Flow into existing projects Domain‑specific GenAI solutions for DRO...Long term contractTemporary workFixed term contractFlexible hours- ...Senior Business Analyst – Lending Domain (Canadian Banking) Required Experience: 7+ years Skills: IT Business Analysis / Business Systems Analysis Banking Lending Domain (Retail Lending, Commercial Lending, Mortgages, Loan Servicing, Credit Products) Lending...Contract work
- ...Business Systems Analyst (Intermediate) – Credit Cards Domain Role: Business Systems Analyst (BSA) Openings: 2 Work Model: Hybrid (4 days/week in office) Overview We are seeking an experienced Business Systems Analyst with strong expertise in the...Contract workWork at office
$50 per hour
...Role Responsibilities Evaluate the accuracy of AI models in... ...understanding within the Surveying & GIS domain. Author complex, grounded... ...AI model training and evaluation processes. Collaborate with subject matter experts to maintain consistency and...Remote jobContract workSummer work$50 per hour
...Jack Dorsey . Position: Architecture Expert Type: Contract Compensation:... ...hours/week Role Responsibilities Evaluate AI models on visual document understanding in the Architecture domain. Author complex, grounded tasks with clear...Remote jobContract workSummer work$90 - $110 per hour
...Position: Finance & Banking Expert Type: Contract... ...produce accurate outputs. Evaluate AI models on professional document... ...within the Finance & Banking domain. Develop complex, multi-step... .... Create objective evaluation rubrics for AI-generated content...Remote jobContract workSummer work
Do you want to receive more vacancies?
Subscribe and receive similar vacancies to Operations Evaluator - Domain Expert. Be the first to apply!
Related searches
- chargé de projet estimateur Toronto, ON
- test evaluator Toronto, ON
- admissions evaluator Toronto, ON
- remote social media evaluator Toronto, ON
- évaluateur-estimateur Toronto, ON
- estimateur automobile Toronto, ON
- estimateur en bâtiment Toronto, ON
- estimateur structure Toronto, ON
- expert automobile Toronto, ON
- expert batiment Toronto, ON