Data Science Expert - Evaluation Specialist
$120 - $170 per hourMercor
Toronto, ON
- Remote job
About the job
Mercor connects elite creative and technical talent with leading AI research labs. Headquartered in San Francisco, our investors include Benchmark , General Catalyst , Peter Thiel , Adam D'Angelo , Larry Summers , and Jack Dorsey .
Position: Data Science Expert
Type: Contract
Compensation: $120–$170/hour
Location: Remote
Role Responsibilities
- Design precise, task-specific grading criteria for real-world data science deliverables, including analyses, models, dashboards, and experiment readouts.
- Score AI-generated and human work samples against established criteria with detailed, well-reasoned written justifications.
- Apply consistent, evidence-based judgment to ensure scores are reproducible and defensible.
- Incorporate structured feedback from senior reviewers and iterate quickly on your work.
- Collaborate with AI research teams to improve training data quality and downstream performance.
Qualifications
Must-Have
- 5+ years of professional data science experience in industry.
- Background in business operations, product, or growth data science at top-tier technology companies.
- Deep fluency in experiment design, A/B testing , metric definition, SQL/Python analysis, and communicating findings to executive stakeholders.
- Exceptionally strong written communication.
- Detail-oriented, consistent, and comfortable having your judgment reviewed and calibrated against peers.
Preferred
- Prior experience with AI training , evaluation, or human-data projects.
Application Process (Takes 20–30 mins to complete)
- Submit your resume or relevant technical background to get started.
- Qualified applicants may be asked to complete a brief technical assessment or submit additional information.
Resources & Support
- For details about the interview process and platform information, please check:
- For any help or support, reach out to: Show email
PS: Our team reviews applications daily. Please complete your AI interview and application steps to be considered for this opportunity.
Vacancy posted a month ago
Similar jobs that could be interesting for youBased on the Data Science Expert - Evaluation Specialist in Toronto, ON vacancy
$100 - $150 per hour
...Jack Dorsey . Position: Data Scientist Talent Network... ...specific grading criteria for data science deliverables, including exploratory... ...A/B test write-ups . Evaluate AI-generated or human-... ...detailed written justifications for evaluations and scores to maintain...SuggestedFull timeContract workSummer workRemote workMercor
Toronto, ONmore than 2 months ago$60 - $70 per hour
...Benchmark , General Catalyst , Peter Thiel , Adam D'Angelo , Larry Summers , and Jack Dorsey . Position: Data Science and Analytics Experts Type: Contract Compensation: $60–$70/hour Location: Remote Role Responsibilities...SuggestedFull timeContract workSummer workRemote workMercor
Toronto, ONa month ago$120 - $170 per hour
...Angelo , Larry Summers , and Jack Dorsey . Position: Data Science Expert Type: Contract Compensation: $120–$170/... ...Preferred ~ Prior experience with AI training , evaluation, or human-data projects. Application Process (Takes 20...SuggestedFull timeContract workSummer workRemote workMercor
Toronto, ONmore than 2 months ago$100 - $150 per hour
...Angelo , Larry Summers , and Jack Dorsey . Position: Data Science Expert Type: Contract Compensation: $100–$150/... ...Location: Remote Role Responsibilities Evaluate AI-generated artifacts for usability in data science contexts...SuggestedRemote jobContract workSummer workWork at office$100 - $150 per hour
...Larry Summers , and Jack Dorsey . Position: Software Engineer Expert Type: Contract Compensation: $100–$150/hour Location: Remote Role Responsibilities Evaluate AI-generated artifacts for usability in a software engineering...SuggestedContract workSummer workWork at officeRemote workMercor
Toronto, ON19 days ago$80 - $120 per hour
...Benchmark , General Catalyst , Peter Thiel , Adam D'Angelo , Larry Summers , and Jack Dorsey . Position: Data quality / CRM operations Evaluator Type: Contract Compensation: $80–$120/hour Location: Remote Role Responsibilities...Full timeContract workSummer workWork at officeRemote workMercor
Toronto, ONmore than 2 months ago$150k - $180k per year
...The position of Senior Consultant - Data Science & AI is within the TTEC Digital Analytics... ...AI Solutions: Architect, fine-tune, and evaluate enterprise Gen AI systems—including Retrieval... ..., automated deployment pipelines, LLM evaluation, and model monitoring tools ~ Personal...Long term contractFull timeFlexible hoursTtec Digital
Toronto, ON17 days ago$80 - $120 per hour
...Jack Dorsey . Position: Privacy / regulatory compliance Evaluator Type: Contract Compensation: $80–$120/hour... ...outputs. Collaborate with AI research teams to enhance training data quality. Work independently and asynchronously to meet deadlines...Full timeContract workSummer workWork at officeRemote workFlexible hoursMercor
Toronto, ONmore than 2 months ago$68.55 - $82.26 per hour
...RQ00745 -Requisition TitleData Science Developer Location:... ...infrastructure that turns public health data into timely analytics,... ...Data Engineer / Data Science Specialist builds and automates the data... ...qualifications. The client will evaluate candidates based on both rate...Full timeSource Code
Toronto, ONmore than 2 months ago$78.95k - $118.42k per year
...We seek a talented and driven Data Scientist passionate about accelerating... ...to our mission of open science and pre-competitive drug... ...or in collaboration with other experts in the Haibe-Kains and Schapira... ...preprocessing through model training, evaluation, and production deployment....RemplacementPermanent employmentFull timeInternshipManual laborFlexible hoursShift workDay shiftUniversity Health Network
Toronto, ONa month ago$90 - $110 per hour
...Larry Summers , and Jack Dorsey . Position: Software & Data Science Expert Type: Contract Compensation: $90–$110/hour... ...to test AI reasoning over technical documentation. Evaluate AI models on professional document understanding. Develop...Full timeContract workSummer workRemote workMercor
Toronto, ONmore than 2 months ago$100 - $150 per hour
...Position: Sales Engineering Expert Type: Contract... ...supporting proofs of concept, and evaluating product fit and technical risk... ...infrastructure, cybersecurity, data platforms, developer tools, or... ...experience with AI training , evaluation, or human-data projects....Full timeContract workSummer workRemote workMercor
Toronto, ONmore than 2 months ago$80 - $120 per hour
...Catalyst , Peter Thiel , Adam D'Angelo , Larry Summers , and Jack Dorsey . Position: Privacy / regulatory compliance Evaluator Type: Contract Compensation: $80–$120/hour Location: Remote Role Responsibilities Evaluate...Full timeContract workSummer workWork at officeRemote workMercor
Toronto, ONmore than 2 months ago$80 - $120 per hour
..., General Catalyst , Peter Thiel , Adam D'Angelo , Larry Summers , and Jack Dorsey . Position: Legal / compliance Evaluator Type: Contract Compensation: $80–$120/hour Location: Remote Role Responsibilities Evaluate...Full timeContract workSummer workWork at officeRemote workMercor
Toronto, ONmore than 2 months ago$60 - $80 per hour
...General Catalyst , Peter Thiel , Adam D'Angelo , Larry Summers , and Jack Dorsey . Position: Business Domain Expert — AI Model Evaluation Pilot (Admin, Marketing, HR, Accounting) Type: Contract Compensation: $60–$80/hour Location:...Hourly payFull timeContract workFor contractorsSummer workRemote workMercor
Toronto, ONa month ago$80 - $120 per hour
...Catalyst , Peter Thiel , Adam D'Angelo , Larry Summers , and Jack Dorsey . Position: Customer success / support operations Evaluator Type: Contract Compensation: $80–$120/hour Location: Remote Role Responsibilities Evaluate...Full timeContract workSummer workWork at officeRemote workMercor
Toronto, ONa month ago$70k - $73.85k per year
...vacancy. Objective: The Research and Evaluation Specialist will lead the analysis and development... ...organizes, analyzes, and communicates data in the form of research findings to... ...to social research; evaluation; social sciences graduate preferred Minimum of 3 years...Full timeInternshipOfifc
Toronto, ONmore than 2 months ago$45 - $48 per hour
...members belong and have the opportunity to thrive. Lyft’s Data Science Team builds mathematical models underpinning the platform’s core... ...and ML models to enable better decisions, and developing and evaluating algorithmic policies in our pricing, dispatch, and incentives...Hourly paySummer workInternshipSummer internshipWork at office3 days per weekLyft
Toronto, ON26 days ago$80 per hour
...Summers , and Jack Dorsey . Position: Insurance Actuarial Specialists Type: Contract Compensation: $80/hour... ...practice area). ~ Understand how methods, assumptions, and data drive pricing, reserves, and valuation. ~ Ability to select...Full timeContract workSummer workImmediate startRemote workMercor
Toronto, ON20 days ago$14 - $42 per hour
...Larry Summers , and Jack Dorsey . Position: Music & Lyrics Expert - Hindi Type: Contract Compensation: $14–$42/... ...Commitment: 20+ hours/week Role Responsibilities Evaluate AI-generated music across various genres. Rate against detailed...Full timeContract workSummer workImmediate startRemote workMercor
Toronto, ONa month ago$70 - $100 per hour
...Larry Summers , and Jack Dorsey . Position: Office-Suite Experts Type: Contract Compensation: $70–$100/hour... ...Commitment: 20–40 hours/week Role Responsibilities Evaluate AI-generated office artifacts in Excel , Word , and...Full timeContract workSummer workWork at officeImmediate startRemote workMercor
Toronto, ONmore than 2 months ago$15 - $20 per hour
...Responsibilities Conduct fact-checking using trusted public sources and external tools . Generate high-quality human evaluation data by identifying response strengths, areas for improvement, and factual inaccuracies. Assess reasoning quality, clarity, tone,...Full timeContract workSummer workRemote workMercor
Toronto, ONmore than 2 months ago$85 - $93 per hour
...Summers , and Jack Dorsey . Position: Basic Medical Sciences Expert Type: Contract Compensation: $85–$93/hour... ...Responsibilities Author and review technical tasks to evaluate AI model performance on basic medical sciences problems....Hourly payFull timeContract workFor contractorsSummer workRemote workFlexible hoursMercor
Toronto, ONmore than 2 months ago$55k per year
...Product Support Specialist (Life Sciences) ➡️ Love being the go-to expert when others need answers and genuinely enjoy solving complex problems for customers? ➡️ Energized by digging deep into products, data, and workflows to help users get real value from powerful technology...Full timeWork at officeLocal areaHome office1 day per weekPatsnap
Toronto, ONmore than 2 months ago$80 per hour
...'Angelo , Larry Summers , and Jack Dorsey . Position: Data Engineer (Coding Agent Experience) Type: Contract... ...Responsibilities Use frontier AI coding agents to complete and evaluate complex data engineering tasks. Review model-generated...Full timeContract workSummer workRemote workMercor
Toronto, ONmore than 2 months ago$80 per hour
...Benchmark , General Catalyst , Peter Thiel , Adam D'Angelo , Larry Summers , and Jack Dorsey . Position: Life & Annuity Specialist Type: Contract Compensation: $80/hour Location: Remote Commitment: 20+ hours/week Role...Hourly payWeekly payLong term contractRemplacementFull timeContract workFor contractorsSummer workImmediate startRemote workMercor
Toronto, ON5 days ago$80 - $160 per hour
...Provide structured feedback to enhance document relevance and impact. Utilize strong analytical skills to contribute to high-impact evaluation projects. Collaborate with teams to improve document standards and guidelines. Work independently and asynchronously to...Full timeContract workSummer workRemote workMercor
Toronto, ONa month ago$15 - $20 per hour
...include Benchmark , General Catalyst , Peter Thiel , Adam D'Angelo , Larry Summers , and Jack Dorsey . Position: Video Evaluation Generalist Type: Contract Compensation: $15–$20/hour Location: Remote Commitment: 20 hours/...Full timeContract workSummer workRemote workMercor
Toronto, ONa month ago$65 - $75 per hour
...Dorsey . Position: Source Security Specialist Type: Contract... ...benign, dual-use, and adversarial. Evaluate AI model responses against a defined policy... ...correct responses. Collaborate with experts to draw reliable lines between routine and...Full timeContract workSummer workRemote workMercor
Toronto, ON26 days ago$30 - $40 per hour
...Generalist Type: Contract Compensation: $30–$40/hour Location: Remote Role Responsibilities Evaluate the functionality and performance of Blender on Windows systems. Capture high-resolution screenshots exceeding 2.5 megapixels...Full timeContract workSummer workRemote workMercor
Toronto, ON28 days ago
Do you want to receive more vacancies?
Subscribe and receive similar vacancies to Data Science Expert - Evaluation Specialist. Be the first to apply!
Related searches