Business Operations Expert - Evaluator
$70 - $100 per hourMercor
Toronto, ON
- Remote job
About the job
Mercor connects elite creative and technical talent with leading AI research labs. Headquartered in San Francisco, our investors include Benchmark , General Catalyst , Peter Thiel , Adam D'Angelo , Larry Summers , and Jack Dorsey .
Position: Business Domain Expert — AI Model Evaluation Pilot (Admin, Marketing, HR, Accounting)
Type: Contract
Compensation: $70–$100/hour
Location: Remote
Role Responsibilities
- Solve one realistic, domain-specific task using the provided source files.
- Review five model-generated outputs for the task. Rank them 1–5 with no ties, score each 0–100, and write evidence-based rationales citing specific parts of the output.
- Provide structured feedback on the task and study design.
- Work independently to meet deadlines within a short turnaround window.
- Use professional judgment as no rubric or golden answer is provided. LLMs or AI assistants are prohibited at every stage.
Qualifications
Must-Have
- 5+ years of hands-on professional experience in Admin , Marketing , Human Resources , or Accounting .
- Currently or recently practicing to judge work as a working professional would.
- Ability to write clear, specific, evidence-grounded rationales for judgments.
- Reliable within a short turnaround window.
Application Process (Takes 20–30 mins to complete)
- Upload resume
- AI interview based on your resume
- Submit form
Resources & Support
- For details about the interview process and platform information, please check:
- For any help or support, reach out to: Show email
PS: Our team reviews applications daily. Please complete your AI interview and application steps to be considered for this opportunity.
Vacancy posted more than 2 months ago
Similar jobs that could be interesting for youBased on the Business Operations Expert - Evaluator in Toronto, ON vacancy
$60 - $80 per hour
...Benchmark , General Catalyst , Peter Thiel , Adam D'Angelo , Larry Summers , and Jack Dorsey . Position: Business Domain Expert — AI Model Evaluation Pilot (Admin, Marketing, HR, Accounting) Type: Contract Compensation: $60–$80/hour...SuggestedRemote jobHourly payContract workFor contractorsSummer work$80 - $120 per hour
...Summers , and Jack Dorsey . Position: General business strategy / management Evaluator Type: Contract Compensation: $80–$120... ...model performance . Collaborate with subject matter experts to ensure consistency, relevance, and coverage across...SuggestedRemote jobContract workSummer workWork at office$80 - $120 per hour
...General Catalyst , Peter Thiel , Adam D'Angelo , Larry Summers , and Jack Dorsey . Position: General business strategy / management Evaluator Type: Contract Compensation: $80–$120/hour Location: Remote Role Responsibilities...SuggestedRemote jobContract workSummer workWork at officeFlexible hours$80 - $120 per hour
..., General Catalyst , Peter Thiel , Adam D'Angelo , Larry Summers , and Jack Dorsey . Position: Finance operations / audit support Evaluator Type: Contract Compensation: $80–$120/hour Location: Remote Role Responsibilities...SuggestedRemote jobContract workSummer workWork at office$80 - $120 per hour
...Larry Summers , and Jack Dorsey . Position: Healthcare operations Evaluator Type: Contract Compensation: $80–$120/... ...independently and asynchronously to ensure high-quality evaluations. Qualifications Must-Have ~5+ years of relevant...SuggestedRemote jobContract workSummer workWork at office$80 - $120 per hour
...Benchmark , General Catalyst , Peter Thiel , Adam D'Angelo , Larry Summers , and Jack Dorsey . Position: Healthcare operations Evaluator Type: Contract Compensation: $80–$120/hour Location: Remote Role Responsibilities...Remote jobContract workSummer workWork at office$30 per hour
...Larry Summers , and Jack Dorsey . Position: Hindi Linguistic Experts Type: Contract Compensation: $30/hour Location: Remote Role Responsibilities Evaluate the quality and accuracy of LLM-generated Hindi text across various...Remote jobContract workSummer work$150 per hour
...Larry Summers , and Jack Dorsey . Position: Healthcare Expert Type: Contract Compensation: $150/hour... ...Commitment: 10–20 hours/week Role Responsibilities Evaluate and compare AI-generated responses to challenging medical problems...Remote jobContract workSummer workImmediate startFlexible hours$100 - $150 per hour
...Larry Summers , and Jack Dorsey . Position: Data Science Expert Type: Contract Compensation: $100–$150/hour Location: Remote Role Responsibilities Evaluate AI-generated artifacts for usability in data science contexts....Remote jobContract workSummer workWork at office- ...payments. Please review the payment terms for each project. PLEASE NOTE: We collect, retain and use personal data for our professional business purposes, including notifying you of opportunities that may be of interest and sharing with our affiliates. We limit the personal...Hourly payContract workFreelanceInternshipRemote workFlexible hours
$87.8k - $116.75k per year
...Identify deficiencies in contractor operations and document any disputes regarding contract... ...improvements, policy development, and problem evaluation. ~ Demonstrated ability to build and... ...of stakeholders and subject matter experts. ~ Knowledge of government...Permanent employmentFull timeContract workTemporary workFor contractorsInternshipMonday to fridayShift work$70k - $80k per year
...with the premier programme, project and cost management offering in markets around the world. Job Description The Business Operations Partner is a key support role within the team, providing a hybrid of executive support, operational coordination, marketing/proposal...Full timeFlexible hours$50 per hour
...Position: English Language and Literature Expert Type: Contract... ...Remote Role Responsibilities Evaluate the quality and accuracy of LLM-generated... ...writing, linguistic annotation, language evaluation, or NLP data work in English. Strong...Remote jobContract workSummer work$100 - $150 per hour
...Larry Summers , and Jack Dorsey . Position: Software Engineer Expert Type: Contract Compensation: $100–$150/hour Location: Remote Role Responsibilities Evaluate AI-generated artifacts for usability in a software engineering...Remote jobContract workSummer workWork at office$150k per year
...Job Responsibility: Join The Forever Group as the Director of Business Operations and Finance and unlock a world of opportunity! With 30 years of industry expertise, we are a Canadian powerhouse, known for our entrepreneurial drive and dedication to crafting top-notch consumer...Full timeInternshipWorldwideRelocationOverseasMonday to fridayFlexible hours$80 - $120 per hour
...Catalyst , Peter Thiel , Adam D'Angelo , Larry Summers , and Jack Dorsey . Position: Data analysis / quantitative readouts Evaluator Type: Contract Compensation: $80–$120/hour Location: Remote Role Responsibilities Evaluate...Remote jobContract workSummer workWork at office$100 - $150 per hour
...Position: Sales Engineering Expert Type: Contract... ...supporting proofs of concept, and evaluating product fit and technical risk... ...work to both technical and business stakeholders. ~ Detail-oriented... ...with AI training , evaluation, or human-data projects....Remote jobContract workSummer work$70 per hour
...General Catalyst , Peter Thiel , Adam D'Angelo , Larry Summers , and Jack Dorsey . Position: Mathematics PhD Coding Experts Type: Contract Compensation: $70/hour Location: Remote Duration: 6 weeks Commitment: 20+...Remote jobContract workSummer workImmediate start$80 per hour
...Catalyst , Peter Thiel , Adam D'Angelo , Larry Summers , and Jack Dorsey . Position: Policy Administration & Insurance Operations Manager Type: Contract Compensation: $80/hour Location: Remote Commitment: 20–40 hours/week...Remote jobContract workSummer workImmediate start$120 - $170 per hour
...Dorsey . Position: Data Science Expert Type: Contract Compensation... ...in industry. ~ Background in business operations, product, or growth data science at top... ...Prior experience with AI training , evaluation, or human-data projects. Application...Remote jobContract workSummer work$120 - $170 per hour
...Dorsey . Position: Data Science Expert Type: Contract Compensation... ...in industry. ~ Background in business operations, product, or growth data science at top... ...Prior experience with AI training , evaluation, or human-data projects. Application...Remote jobContract workSummer work$80 - $120 per hour
...Catalyst , Peter Thiel , Adam D'Angelo , Larry Summers , and Jack Dorsey . Position: Biology / environmental science Evaluator Type: Contract Compensation: $80–$120/hour Location: Remote Role Responsibilities Evaluate...Remote jobContract workSummer workWork at office$80 - $120 per hour
...Dorsey . Position: Market research / competitive intelligence Evaluator Type: Contract Compensation: $80–$120/hour... ...AI model performance . Collaborate with subject matter experts to ensure consistency and relevance across datasets. Qualifications...Remote jobContract workSummer workWork at office$100 - $150 per hour
...Larry Summers , and Jack Dorsey . Position: B2B Sales Expert Type: Contract Compensation: $100–$150/hour... ...independently and asynchronously to meet deadlines while improving evaluation processes. Qualifications Must-Have ~5+ years...Remote jobContract workSummer work$100 - $150 per hour
...Larry Summers , and Jack Dorsey . Position: B2B Sales Expert Type: Contract Compensation: $100–$150/hour... ...Preferred ~ Prior experience with AI training , evaluation, or human-data projects. Application Process (Takes 20–30...Remote jobContract workSummer work$80 - $150 per hour
..., Larry Summers , and Jack Dorsey . Position: Design Expert Type: Contract Compensation: $80–$150/hour... ...Preferred ~ Prior experience with AI training , evaluation, or human-data projects. Application Process (Takes 20–30...Remote jobContract workSummer work$50 - $70 per hour
...Larry Summers , and Jack Dorsey . Position: Generalist Expert (US/Canada) Type: Contract Compensation: $50–$... ...Location: Remote Role Responsibilities Review and evaluate documents, slides, spreadsheets, and similar materials for...Remote jobContract workSummer workFlexible hours$80 - $120 per hour
...General Catalyst , Peter Thiel , Adam D'Angelo , Larry Summers , and Jack Dorsey . Position: Healthcare / clinical Evaluator Type: Contract Compensation: $80–$120/hour Location: Remote Role Responsibilities Evaluate...Remote jobContract workSummer workWork at office$71.4k - $105.66k per year
...the play! Position Overview Zynga, the global leader in mobile games, is currently seeking a full-time Associate Manager, Business Operations . We are looking for a highly organized, self-motivated operator who excels at cross-functional communication, process...Full timeLocal area$14 per hour
...Larry Summers , and Jack Dorsey . Position: Music Audio Expert - Telugu Type: Contract Compensation: $14/... ...Commitment: 10+ hours/week Role Responsibilities Evaluate AI model output lyrics , voice generation, and other standards...Remote jobContract workSummer workImmediate start
Do you want to receive more vacancies?
Subscribe and receive similar vacancies to Business Operations Expert - Evaluator. Be the first to apply!
Related searches
- business systems consultant Toronto, ON
- business sales specialist Toronto, ON
- emergency management consultant Toronto, ON
- project management consultant Toronto, ON
- small business associate Toronto, ON
- small business advisor Toronto, ON
- junior business consultant Toronto, ON
- change management consultant Toronto, ON
- change management advisor Toronto, ON
- business continuity specialist Toronto, ON