Business Operations Expert - Evaluator
$70 - $100 per hourMercor
Toronto, ON
- Remote job
About the job
Mercor connects elite creative and technical talent with leading AI research labs. Headquartered in San Francisco, our investors include Benchmark , General Catalyst , Peter Thiel , Adam D'Angelo , Larry Summers , and Jack Dorsey .
Position: Business Domain Expert — AI Model Evaluation Pilot (Admin, Marketing, HR, Accounting)
Type: Contract
Compensation: $70–$100/hour
Location: Remote
Role Responsibilities
- Solve one realistic, domain-specific task using the provided source files.
- Review five model-generated outputs for the task. Rank them 1–5 with no ties, score each 0–100, and write evidence-based rationales citing specific parts of the output.
- Provide structured feedback on the task and study design.
- Work independently to meet deadlines within a short turnaround window.
- Use professional judgment as no rubric or golden answer is provided. LLMs or AI assistants are prohibited at every stage.
Qualifications
Must-Have
- 5+ years of hands-on professional experience in Admin , Marketing , Human Resources , or Accounting .
- Currently or recently practicing to judge work as a working professional would.
- Ability to write clear, specific, evidence-grounded rationales for judgments.
- Reliable within a short turnaround window.
Application Process (Takes 20–30 mins to complete)
- Upload resume
- AI interview based on your resume
- Submit form
Resources & Support
- For details about the interview process and platform information, please check:
- For any help or support, reach out to: Show email
PS: Our team reviews applications daily. Please complete your AI interview and application steps to be considered for this opportunity.
Vacancy posted 23 days ago
Similar jobs that could be interesting for youBased on the Business Operations Expert - Evaluator in Toronto, ON vacancy
$60 - $80 per hour
...Benchmark , General Catalyst , Peter Thiel , Adam D'Angelo , Larry Summers , and Jack Dorsey . Position: Business Domain Expert — AI Model Evaluation Pilot (Admin, Marketing, HR, Accounting) Type: Contract Compensation: $60–$80/hour...SuggestedHourly payFull timeContract workFor contractorsSummer workRemote work$80 - $120 per hour
..., General Catalyst , Peter Thiel , Adam D'Angelo , Larry Summers , and Jack Dorsey . Position: Finance operations / audit support Evaluator Type: Contract Compensation: $80–$120/hour Location: Remote Role Responsibilities...SuggestedRemote jobContract workSummer workWork at office$80 - $120 per hour
...Benchmark , General Catalyst , Peter Thiel , Adam D'Angelo , Larry Summers , and Jack Dorsey . Position: Healthcare operations Evaluator Type: Contract Compensation: $80–$120/hour Location: Remote Role Responsibilities...SuggestedRemote jobContract workSummer workWork at office$80 - $120 per hour
..., General Catalyst , Peter Thiel , Adam D'Angelo , Larry Summers , and Jack Dorsey . Position: Operations / inventory / capacity planning Evaluator Type: Contract Compensation: $80–$120/hour Location: Remote Role Responsibilities...SuggestedRemote jobContract workSummer workWork at office$80 - $120 per hour
...Benchmark General Catalyst Peter Thiel Adam DAngelo Larry Summers and Jack Dorsey . Position: Operations / inventory / capacity planning Evaluator Type: Contract Compensation: $80$120/hour Location: Remote Role...SuggestedFull timeContract workSummer workWork at officeRemote work$80 - $120 per hour
..., General Catalyst , Peter Thiel , Adam D'Angelo , Larry Summers , and Jack Dorsey . Position: Finance operations / audit support Evaluator Type: Contract Compensation: $80–$120/hour Location: Remote Role Responsibilities...Remote jobContract workSummer workWork at office$80 - $120 per hour
...Larry Summers , and Jack Dorsey . Position: Healthcare operations Evaluator Type: Contract Compensation: $80–$120/... ...independently and asynchronously to ensure high-quality evaluations. Qualifications Must-Have ~5+ years of relevant...Remote jobContract workSummer workWork at office$80 - $120 per hour
...General Catalyst , Peter Thiel , Adam D'Angelo , Larry Summers , and Jack Dorsey . Position: Data quality / CRM operations Evaluator Type: Contract Compensation: $80–$120/hour Location: Remote Role Responsibilities Evaluate...Full timeContract workSummer workWork at officeRemote work$100 - $150 per hour
...Position: Sales Engineering Expert Type: Contract... ...supporting proofs of concept, and evaluating product fit and technical risk... ...work to both technical and business stakeholders. ~ Detail-oriented... ...with AI training , evaluation, or human-data projects....Full timeContract workSummer workRemote work$80 - $120 per hour
...Catalyst , Peter Thiel , Adam D'Angelo , Larry Summers , and Jack Dorsey . Position: Customer success / support operations Evaluator Type: Contract Compensation: $80–$120/hour Location: Remote Role Responsibilities Evaluate...Full timeContract workSummer workWork at officeRemote work$120 - $170 per hour
...Dorsey . Position: Data Science Expert Type: Contract Compensation... ...in industry. ~ Background in business operations, product, or growth data science at top... ...Prior experience with AI training , evaluation, or human-data projects. Application...Full timeContract workSummer workRemote work$120 - $170 per hour
...Dorsey . Position: Data Science Expert Type: Contract Compensation... ...in industry. ~ Background in business operations, product, or growth data science at top... ...Prior experience with AI training , evaluation, or human-data projects. Application...Full timeContract workSummer workRemote work$50 per hour
...Position: English Language and Literature Expert Type: Contract... ...Remote Role Responsibilities Evaluate the quality and accuracy of LLM-generated... ...writing, linguistic annotation, language evaluation, or NLP data work in English. Strong...Remote jobContract workSummer work$70 per hour
...General Catalyst , Peter Thiel , Adam D'Angelo , Larry Summers , and Jack Dorsey . Position: Mathematics PhD Coding Experts Type: Contract Compensation: $70/hour Location: Remote Duration: 6 weeks Commitment: 20+...Remote jobContract workSummer workImmediate start$50 - $70 per hour
...Larry Summers , and Jack Dorsey . Position: Generalist Expert (US/Canada) Type: Contract Compensation: $50–$... ...Location: Remote Role Responsibilities Review and evaluate documents, slides, spreadsheets, and similar materials for...Remote jobContract workSummer workFlexible hours$80 - $120 per hour
...Catalyst Peter Thiel Adam DAngelo Larry Summers and Jack Dorsey . Position: BI dashboards / performance reporting Evaluator Type: Contract Compensation: $80$120/hour Location: Remote Role Responsibilities Evaluate AI...Full timeContract workSummer workWork at officeRemote work$100 - $150 per hour
...Larry Summers , and Jack Dorsey . Position: B2B Sales Expert Type: Contract Compensation: $100–$150/hour... ...independently and asynchronously to meet deadlines while improving evaluation processes. Qualifications Must-Have ~5+ years...Remote jobContract workSummer work$80 - $120 per hour
...Benchmark General Catalyst Peter Thiel Adam DAngelo Larry Summers and Jack Dorsey . Position: Legal / compliance Evaluator Type: Contract Compensation: $80$120/hour Location: Remote Role Responsibilities Evaluate...Full timeContract workSummer workWork at officeRemote work$80 - $120 per hour
...General Catalyst Peter Thiel Adam DAngelo Larry Summers and Jack Dorsey . Position: IP / trademark / copyright law Evaluator Type: Contract Compensation: $80$120/hour Location: Remote Role Responsibilities Evaluate...Full timeContract workSummer workWork at officeRemote work$80 - $120 per hour
...General Catalyst , Peter Thiel , Adam D'Angelo , Larry Summers , and Jack Dorsey . Position: Healthcare / clinical Evaluator Type: Contract Compensation: $80–$120/hour Location: Remote Role Responsibilities Evaluate...Remote jobContract workSummer workWork at office- ...commercial insurance for Canadian businesses. As a leading InsurTech,... ...you! As a Senior Business Operations Manager, you'll be a key... ...solutions. Become a Technical Expert: Develop a deep... ...the interview could affect our evaluation of your suitability for the role...Remote jobLong term contractFull timeWork at officeWork from homeHome officeFlexible hoursWeekend work
- ...with global brands and emerging businesses to build reputations, create... ...minded Director of Business Operations to help lead the agency's... ...efficiency across the agency. Evaluate and implement technology that... .... Conduct post-project evaluations and identify opportunities for...Full timeSummer workWork at officeFlexible hours
$11 - $19 per hour
...Larry Summers and Jack Dorsey . Position: Music & Lyrics Expert - Bengali Type: Contract Compensation: $11$19... ...on quality creativity prompt adherence and originality. Evaluate whether lyrics sound natural including word choice slang and regional...Full timeContract workSummer workImmediate startRemote work$78.4k - $110k per year
...everything we do across all our diverse businesses: • Curiosity : We do our best work... ...Royalty Department is a Technology Operations and transformation department with the objective... ...an equal opportunity workplace and will evaluate qualified applicants without regard to...Full timeFor contractorsLocal areaWorldwide$80 - $150 per hour
...Larry Summers , and Jack Dorsey . Position: UI / UX Design Expert Type: Contract Compensation: $80–$150/hour... ...Preferred ~ Prior experience with AI training , evaluation, or human-data projects. Application Process (Takes 20–30...Remote jobContract workSummer work$14 per hour
...Larry Summers , and Jack Dorsey . Position: Music Audio Expert - Telugu Type: Contract Compensation: $14/... ...Commitment: 10+ hours/week Role Responsibilities Evaluate AI model output lyrics , voice generation, and other standards...Remote jobContract workSummer workImmediate start$55 - $65 per hour
...Position: Manufacturing Operations Reviewer Type: Contract... ...Role Responsibilities Evaluate AI-generated content in... ...generated scores and provide expert judgment. Collaborate with... ...operations. ~ Near-native business English proficiency. ~ Comfort...Remote jobContract workSummer workShift work$79k - $129k per year
...Job Description The Manager, Business Operations in Product Engineering will support all business operations activities within the Bedrock... ..., you may contact our Human Resources Department at HR.Leave-Expert@thomsonreuters.com . Disability accommodations in the recruiting...Full timeWork at officeLocal areaFlexible hours$45 - $65 per hour
...Role Responsibilities Review and evaluate AI-generated outputs related to... ...institutional policy, and higher education operations. Create realistic scenarios based on... ...asynchronously with research teams to refine evaluation frameworks for education-domain AI ....Full timeContract workSummer workRemote work$80 - $160 per hour
..., Larry Summers , and Jack Dorsey . Position: Operations / Program Management Experts Type: Contract Compensation: $80–$160... ...Provide structured feedback to improve document standards and evaluation processes. Collaborate with teams to enhance...Remote jobContract workSummer work
Do you want to receive more vacancies?
Subscribe and receive similar vacancies to Business Operations Expert - Evaluator. Be the first to apply!
Related searches
- business continuity specialist Toronto, ON
- risk management consultant Toronto, ON
- project management consultant Toronto, ON
- change management advisor Toronto, ON
- senior business consultant Toronto, ON
- emergency management consultant Toronto, ON
- document management consultant Toronto, ON
- business consultant Toronto, ON
- business systems consultant Toronto, ON
- small business associate Toronto, ON