Data Science Expert - Evaluation Specialist
$120 - $170 per hourMercor
Toronto, ON
- Remote job
About the job
Mercor connects elite creative and technical talent with leading AI research labs. Headquartered in San Francisco, our investors include Benchmark , General Catalyst , Peter Thiel , Adam D'Angelo , Larry Summers , and Jack Dorsey .
Position: Data Science Expert
Type: Contract
Compensation: $120–$170/hour
Location: Remote
Role Responsibilities
- Design precise, task-specific grading criteria for real-world data science deliverables, including analyses, models, dashboards, and experiment readouts.
- Score AI-generated and human work samples against established criteria with detailed, well-reasoned written justifications.
- Apply consistent, evidence-based judgment to ensure scores are reproducible and defensible.
- Incorporate structured feedback from senior reviewers and iterate quickly on your work.
- Collaborate with AI research teams to improve training data quality and downstream performance.
Qualifications
Must-Have
- 5+ years of professional data science experience in industry.
- Background in business operations, product, or growth data science at top-tier technology companies.
- Deep fluency in experiment design, A/B testing , metric definition, SQL/Python analysis, and communicating findings to executive stakeholders.
- Exceptionally strong written communication.
- Detail-oriented, consistent, and comfortable having your judgment reviewed and calibrated against peers.
Preferred
- Prior experience with AI training , evaluation, or human-data projects.
Application Process (Takes 20–30 mins to complete)
- Submit your resume or relevant technical background to get started.
- Qualified applicants may be asked to complete a brief technical assessment or submit additional information.
Resources & Support
- For details about the interview process and platform information, please check:
- For any help or support, reach out to: Show email
PS: Our team reviews applications daily. Please complete your AI interview and application steps to be considered for this opportunity.
Vacancy posted 23 days ago
Similar jobs that could be interesting for youBased on the Data Science Expert - Evaluation Specialist in Toronto, ON vacancy
$120 - $170 per hour
...Angelo , Larry Summers , and Jack Dorsey . Position: Data Science Expert Type: Contract Compensation: $120–$170/... ...Preferred ~ Prior experience with AI training , evaluation, or human-data projects. Application Process (Takes 20...SuggestedFull timeContract workSummer workRemote work$120 - $170 per hour
...Angelo , Larry Summers , and Jack Dorsey . Position: Data Science Expert Type: Contract Compensation: $120–$170/... ...Preferred ~ Prior experience with AI training , evaluation, or human-data projects. Application Process (Takes 20...SuggestedFull timeContract workSummer workRemote work$80 - $120 per hour
...Benchmark , General Catalyst , Peter Thiel , Adam D'Angelo , Larry Summers , and Jack Dorsey . Position: Data quality / CRM operations Evaluator Type: Contract Compensation: $80–$120/hour Location: Remote Role Responsibilities...SuggestedFull timeContract workSummer workWork at officeRemote work$100 - $150 per hour
...Jack Dorsey . Position: Data Scientist Talent Network... ...specific grading criteria for data science deliverables, including exploratory... ...A/B test write-ups . Evaluate AI-generated or human-... ...detailed written justifications for evaluations and scores to maintain...SuggestedFull timeContract workSummer workRemote work$60 - $70 per hour
...Benchmark , General Catalyst , Peter Thiel , Adam D'Angelo , Larry Summers , and Jack Dorsey . Position: Data Science and Analytics Experts Type: Contract Compensation: $60–$70/hour Location: Remote Role Responsibilities...SuggestedFull timeContract workSummer workRemote work$80 - $120 per hour
...Benchmark General Catalyst Peter Thiel Adam DAngelo Larry Summers and Jack Dorsey . Position: Data quality / CRM operations Evaluator Type: Contract Compensation: $80$120/hour Location: Remote Role Responsibilities...Full timeContract workSummer workWork at officeRemote work$80 - $120 per hour
...Benchmark , General Catalyst , Peter Thiel , Adam D'Angelo , Larry Summers , and Jack Dorsey . Position: Data quality / CRM operations Evaluator Type: Contract Compensation: $80–$120/hour Location: Remote Role Responsibilities...Remote jobContract workSummer workWork at office$68.55 - $82.26 per hour
...RQ00745 -Requisition TitleData Science Developer Location:... ...infrastructure that turns public health data into timely analytics,... ...Data Engineer / Data Science Specialist builds and automates the data... ...qualifications. The client will evaluate candidates based on both rate...Full time$150 per hour
...Jack Dorsey . Position: Data Scientist Talent Network... ...specific grading criteria for data science deliverables. This includes exploratory... ..., and technical reports. Evaluate AI-generated or human-... ...written justifications for evaluations and scores to maintain...Remote jobContract workSummer work$78.95k - $118.42k per year
...We seek a talented and driven Data Scientist passionate about accelerating... ...to our mission of open science and pre-competitive drug... ...or in collaboration with other experts in the Haibe-Kains and Schapira... ...preprocessing through model training, evaluation, and production deployment....RemplacementPermanent employmentFull timeInternshipManual laborFlexible hoursShift workDay shift$90 - $110 per hour
...Larry Summers , and Jack Dorsey . Position: Software & Data Science Expert Type: Contract Compensation: $90–$110/hour... ...to test AI reasoning over technical documentation. Evaluate AI models on professional document understanding. Develop...Full timeContract workSummer workRemote work$80 - $120 per hour
...Jack Dorsey . Position: Privacy / regulatory compliance Evaluator Type: Contract Compensation: $80–$120/hour... ...outputs. Collaborate with AI research teams to enhance training data quality. Work independently and asynchronously to meet deadlines...Full timeContract workSummer workWork at officeRemote workFlexible hours$60 - $80 per hour
...General Catalyst , Peter Thiel , Adam D'Angelo , Larry Summers , and Jack Dorsey . Position: Business Domain Expert — AI Model Evaluation Pilot (Admin, Marketing, HR, Accounting) Type: Contract Compensation: $60–$80/hour Location:...Hourly payFull timeContract workFor contractorsSummer workRemote work$100 - $150 per hour
...Position: Sales Engineering Expert Type: Contract... ...supporting proofs of concept, and evaluating product fit and technical risk... ...infrastructure, cybersecurity, data platforms, developer tools, or... ...experience with AI training , evaluation, or human-data projects....Full timeContract workSummer workRemote work$80 - $120 per hour
...Catalyst , Peter Thiel , Adam D'Angelo , Larry Summers , and Jack Dorsey . Position: Privacy / regulatory compliance Evaluator Type: Contract Compensation: $80–$120/hour Location: Remote Role Responsibilities Evaluate...Full timeContract workSummer workWork at officeRemote work- ...and solve challenging problems and make significant and lasting impact on Canadians. We are seeking an experienced Manager of Data Science & Engineering to lead our Budget Optimization, Forecasting, and Analytics initiatives in our Media & Advertising domain. There would...Permanent employmentFull timeTemporary workInternship
$80 - $120 per hour
..., General Catalyst , Peter Thiel , Adam D'Angelo , Larry Summers , and Jack Dorsey . Position: Legal / compliance Evaluator Type: Contract Compensation: $80–$120/hour Location: Remote Role Responsibilities Evaluate...Full timeContract workSummer workWork at officeRemote work$50 per hour
...Position: English Language and Literature Expert Type: Contract... ...Remote Role Responsibilities Evaluate the quality and accuracy of LLM-generated... ...writing, linguistic annotation, language evaluation, or NLP data work in English. Strong understanding...Remote jobContract workSummer work- ...new phase of growth and modernization, with a focus on stronger data visibility, better decision-making, improved operational discipline... .... About the Role We are creating a new Head of Data Science & Analytics role to build and lead the data function across Big Viking...Full timeWork at office
$70k - $73.85k per year
...vacancy. Objective: The Research and Evaluation Specialist will lead the analysis and development... ...organizes, analyzes, and communicates data in the form of research findings to... ...to social research; evaluation; social sciences graduate preferred Minimum of 3 years...Full timeInternship$80 - $120 per hour
...Benchmark , General Catalyst , Peter Thiel , Adam D'Angelo , Larry Summers , and Jack Dorsey . Position: Data quality / CRM operations Evaluator Type: Contract Compensation: $80–$120/hour Location: Remote Role Responsibilities...Remote jobContract workSummer workWork at office$80 - $120 per hour
...and Jack Dorsey . Position: Humanities / arts / culture Evaluator Type: Contract Compensation: $80–$120/hour... ...AI model performance . Collaborate with subject matter experts to ensure consistency and quality. Work independently and...Remote jobContract workSummer workWork at office$80 - $120 per hour
..., Peter Thiel , Adam D'Angelo , Larry Summers , and Jack Dorsey . Position: Public-sector procurement / RFI response Evaluator Type: Contract Compensation: $80–$120/hour Location: Remote Role Responsibilities Evaluate...Remote jobContract workSummer workWork at office$14 - $42 per hour
...Larry Summers , and Jack Dorsey . Position: Music & Lyrics Expert - Hindi Type: Contract Compensation: $14–$42/... ...Commitment: 20+ hours/week Role Responsibilities Evaluate AI-generated music across various genres. Rate against detailed...Full timeContract workSummer workImmediate startRemote work$70 - $100 per hour
...Larry Summers , and Jack Dorsey . Position: Office-Suite Experts Type: Contract Compensation: $70–$100/hour... ...Commitment: 20–40 hours/week Role Responsibilities Evaluate AI-generated office artifacts in Excel , Word , and...Full timeContract workSummer workWork at officeImmediate startRemote work$85 - $93 per hour
...Summers , and Jack Dorsey . Position: Basic Medical Sciences Expert Type: Contract Compensation: $85–$93/hour... ...Responsibilities Author and review technical tasks to evaluate AI model performance on basic medical sciences problems....Hourly payFull timeContract workFor contractorsSummer workRemote workFlexible hours$15 - $20 per hour
...Responsibilities Conduct fact-checking using trusted public sources and external tools . Generate high-quality human evaluation data by identifying response strengths, areas for improvement, and factual inaccuracies. Assess reasoning quality, clarity, tone,...Full timeContract workSummer workRemote work$50 - $70 per hour
...Larry Summers , and Jack Dorsey . Position: Generalist Expert (US/Canada) Type: Contract Compensation: $50–$... ...Location: Remote Role Responsibilities Review and evaluate documents, slides, spreadsheets, and similar materials for...Remote jobContract workSummer workFlexible hours$15 - $20 per hour
...include Benchmark , General Catalyst , Peter Thiel , Adam D'Angelo , Larry Summers , and Jack Dorsey . Position: Video Evaluation Generalist Type: Contract Compensation: $15–$20/hour Location: Remote Commitment: 20 hours/...Full timeContract workSummer workRemote work$100 - $150 per hour
...Jack Dorsey . Position: B2B Sales Expert Type: Contract Compensation... ...to meet deadlines while improving evaluation processes. Qualifications Must-Have... ...with AI training , evaluation, or human-data projects. Application Process (Takes...Remote jobContract workSummer work
Do you want to receive more vacancies?
Subscribe and receive similar vacancies to Data Science Expert - Evaluation Specialist. Be the first to apply!
Related searches