Data Science Expert - AI Evaluation

$120 - $170 per hour

Full-time

Mercor

About the job

Mercor connects elite creative and technical talent with leading AI research labs. Headquartered in San Francisco, our investors include Benchmark , General Catalyst , Peter Thiel , Adam D'Angelo , Larry Summers , and Jack Dorsey .

Position: Data Science Expert

Type: Contract

Compensation: $120–$170/hour

Location: Remote

Role Responsibilities

Design precise, task-specific grading criteria for real-world data science deliverables such as analyses, models, dashboards, and experiment readouts.
Score AI-generated and human work samples against criteria with detailed written justifications for every score.
Apply consistent, evidence-based judgment to ensure scores are reproducible and defensible.
Incorporate structured feedback from senior reviewers and iterate quickly on your work.
Work independently and asynchronously to meet deadlines while improving AI model performance .

Qualifications

Must-Have

5+ years of professional data science experience in industry.
Background in business operations, product, or growth data science at top-tier technology companies.
Deep fluency in experiment design and A/B testing, metric definition, SQL/Python analysis, and communicating findings to executive stakeholders.
Exceptionally strong written communication.
Detail-oriented, consistent, and comfortable having your judgment reviewed and calibrated against peers.

Preferred

Prior experience with AI training , evaluation, or human-data projects.

Application Process (Takes 20–30 mins to complete)

Submit your resume or relevant technical background to get started.
Qualified applicants may be asked to complete a brief technical assessment or submit additional information.

Resources & Support

For details about the interview process and platform information, please check:
For any help or support, reach out to: View email address on jobs.jobcopilot.com

PS: Our team reviews applications daily. Please complete your AI interview and application steps to be considered for this opportunity.

Apply

Vacancy posted a month ago

Similar jobs that could be interesting for youBased on the Data Science Expert - AI Evaluation in Toronto, ON vacancy

Data Science Evaluator - Expert
$100 - $150 per hour
...Mercor connects elite creative and technical talent with leading AI research labs. Headquartered in San Francisco, our investors... ...Angelo , Larry Summers , and Jack Dorsey . Position: Data Science Expert Type: Contract Compensation: $100–$150/hour...
Suggested
Contract work
Summer work
Work at office
Remote work
Mercor
Toronto, ON
1 day ago
Data Science Expert - Evaluation Specialist
$120 - $170 per hour
...Mercor connects elite creative and technical talent with leading AI research labs. Headquartered in San Francisco, our investors... ...Angelo , Larry Summers , and Jack Dorsey . Position: Data Science Expert Type: Contract Compensation: $120–$170/hour...
Suggested
Full time
Contract work
Summer work
Remote work
Mercor
Toronto, ON
26 days ago
Data Scientist - AI Evaluation
$100 - $150 per hour
...technical talent with leading AI research labs. Headquartered in... ...Jack Dorsey . Position: Data Scientist Talent Network... ...specific grading criteria for data science deliverables, including exploratory... ...A/B test write-ups . Evaluate AI-generated or human-created...
Suggested
Full time
Contract work
Summer work
Remote work
Mercor
Toronto, ON
a month ago
Data Quality Expert - Evaluator
$80 - $120 per hour
...connects elite creative and technical talent with leading AI research labs. Headquartered in San Francisco, our investors... ..., Larry Summers , and Jack Dorsey . Position: Data quality / CRM operations Evaluator Type: Contract Compensation: $80–$120/hour...
Suggested
Full time
Contract work
Summer work
Work at office
Remote work
Mercor
Toronto, ON
a month ago
Data Engineer - AI Model Evaluator
$80 per hour
...creative and technical talent with leading AI research labs. Headquartered in San Francisco... ..., and Jack Dorsey . Position: Data Engineer (Coding Agent Experience) Type... ...frontier AI coding agents to complete and evaluate complex data engineering tasks. Review...
Suggested
Full time
Contract work
Summer work
Remote work
Mercor
Toronto, ON
a month ago
Senior Data Science Analyst
$78.95k - $118.42k per year
...We seek a talented and driven Data Scientist passionate about accelerating... ...through machine learning and AI to join our team at the... ...contribute to our mission of open science and pre-competitive drug discovery... ...or in collaboration with other experts in the Haibe-Kains and Schapira...
Remplacement
Permanent employment
Full time
Internship
Manual labor
Flexible hours
Shift work
Day shift
University Health Network
Toronto, ON
27 days ago
Software Engineer - Data Science
$90 - $110 per hour
...Mercor connects elite creative and technical talent with leading AI research labs. Headquartered in San Francisco, our investors... ...Larry Summers , and Jack Dorsey . Position: Software & Data Science Expert Type: Contract Compensation: $90–$110/hour...
Full time
Contract work
Summer work
Remote work
Mercor
Toronto, ON
more than 2 months ago
AI Evaluation Engineer (QA)
...digital experiences and client partnerships. As a QA / AI Evaluation Engineer, you will join a highly motivated and... ...4+ years in QA / test engineering, with exposure to data/ML systems. ~ Strong Python and data-science techniques for measuring factual grounding and answer...
Full time
Immediate start
Appnovation Technologies
Toronto, ON
27 days ago
Data Evaluator - Fully Remote
$80 - $120 per hour
...Mercor connects elite creative and technical talent with leading AI research labs. Headquartered in San Francisco, our investors... ...Summers , and Jack Dorsey . Position: Software / AI / IT / data Evaluator Type: Contract Compensation: $80–$120/hour...
Contract work
Summer work
Work at office
Remote work
Mercor
Toronto, ON
21 days ago
Software Engineering Evaluator - Expert
$100 - $150 per hour
...creative and technical talent with leading AI research labs. Headquartered in San... ...Dorsey . Position: Software Engineer Expert Type: Contract Compensation... ...Remote Role Responsibilities Evaluate AI-generated artifacts for usability in...
Contract work
Summer work
Work at office
Remote work
Mercor
Toronto, ON
1 day ago
Head of Data Science & Analytics
...and modernization, with a focus on stronger data visibility, better decision-making,... ...operational discipline, and practical adoption of AI across the business. This is a hybrid... ...Role We are creating a new Head of Data Science & Analytics role to build and lead the data...
Full time
Work at office
Big Viking Games
Toronto, ON
more than 2 months ago
AI Data Science Domain Expert
$100 per hour
...spam folder. This is a fully remote position open to candidates based in Canada. Pay: $100$200/hr Role Title: AI Data Science Domain Expert Role Type: Contractor (Part Time) Location: Remote We are engaging AI Data Science Domain Experts to participate in...
Full time
Contract work
Part time
For contractors
Remote work
RemoteJobsOne
Toronto, ON
14 hours ago
RQ00745-Data Science Developer
$68.55 - $82.26 per hour
...RQ00745 -Requisition TitleData Science Developer Location: Toronto... ...that turns public health data into timely analytics, surveillance... ...prioritize use cases and integrate AI techniques (machine learning,... .... The client will evaluate candidates based on both rate expectations...
Full time
Source Code
Toronto, ON
a month ago
Enterprise Data Science Specialist
$60 - $70 per hour
...connects elite creative and technical talent with leading AI research labs. Headquartered in San Francisco, our investors... ...Angelo , Larry Summers , and Jack Dorsey . Position: Data Science and Analytics Experts Type: Contract Compensation: $60–$70/...
Full time
Contract work
Summer work
Remote work
Mercor
Toronto, ON
a month ago
Technical Product Manager, Applied Science AI Products
$125k - $135k per year
...Manager to own how Caseware builds, evaluates, and raises the quality of AI across the platform. You are the product... ...scientists, engineering, domain experts, and external partner firms. You own... ...outcome and the quality bar; the applied science team owns the science underneath it....
Permanent employment
Full time
Work at office
Remote work
Flexible hours
3 days per week
Caseware
Toronto, ON
10 days ago
Sales Engineering Expert - Evaluator
$100 - $150 per hour
...technical talent with leading AI research labs. Headquartered in... ...Position: Sales Engineering Expert Type: Contract... ...supporting proofs of concept, and evaluating product fit and technical risk.... ...infrastructure, cybersecurity, data platforms, developer tools, or...
Full time
Contract work
Summer work
Remote work
Mercor
Toronto, ON
a month ago
Privacy Compliance Expert - Evaluator
$80 - $120 per hour
...Mercor connects elite creative and technical talent with leading AI research labs. Headquartered in San Francisco, our investors... ...Jack Dorsey . Position: Privacy / regulatory compliance Evaluator Type: Contract Compensation: $80–$120/hour...
Full time
Contract work
Summer work
Work at office
Remote work
Mercor
Toronto, ON
more than 2 months ago
Customer Success Evaluator - Expert
$80 - $120 per hour
...Mercor connects elite creative and technical talent with leading AI research labs. Headquartered in San Francisco, our investors... ...Dorsey . Position: Customer success / support operations Evaluator Type: Contract Compensation: $80–$120/hour...
Full time
Contract work
Summer work
Work at office
Remote work
Mercor
Toronto, ON
a month ago
Financial Compliance AI Evaluator
$80 - $120 per hour
...Mercor connects elite creative and technical talent with leading AI research labs. Headquartered in San Francisco, our investors... ...Position: Compliance / regulatory response with financial-services AI Evaluator Type: Contract Compensation: $80–$120/hour...
Full time
Contract work
Summer work
Work at office
Remote work
Mercor
Toronto, ON
more than 2 months ago
Senior Data/AI Engineer
$135k - $210k per year
...Overview: Guidepoint seeks an experienced Data/AI Engineer as an integral member of the... ...Best Practices: Collaborate with data science and engineering teams to establish and implement... ...for LLMOps, including automated evaluation using frameworks like LLM Judges or MLflow...
Full time
Worldwide
Guidepoint
Toronto, ON
more than 2 months ago
Legal Compliance Expert - Evaluator
$80 - $120 per hour
...Mercor connects elite creative and technical talent with leading AI research labs. Headquartered in San Francisco, our investors... ...Summers , and Jack Dorsey . Position: Legal / compliance Evaluator Type: Contract Compensation: $80–$120/hour...
Full time
Contract work
Summer work
Work at office
Remote work
Mercor
Toronto, ON
more than 2 months ago
Business Operations Expert - Evaluator
$60 - $80 per hour
...connects elite creative and technical talent with leading AI research labs. Headquartered in San Francisco, our investors... ...Summers , and Jack Dorsey . Position: Business Domain Expert — AI Model Evaluation Pilot (Admin, Marketing, HR, Accounting) Type:...
Hourly pay
Full time
Contract work
For contractors
Summer work
Remote work
Mercor
Toronto, ON
a month ago
Director, Data & AI (Hybrid)
...Starts Here We’re looking for a Director, Data & AI to lead our Data Engineering, Data & ML Platform, Data Science, and Applied AI teams. You'll own the strategy,... ...experimentation frameworks, guardrail metrics, and evaluation pipelines that drive confident product...
Hourly pay
Long term contract
Full time
Temporary work
Internship
Work at office
Local area
Work from home
Relocation
Flexible hours
Homebase
Toronto, ON
a month ago
Data Science Intern, Algorithms (Summer 2027)
$45 - $48 per hour
...members belong and have the opportunity to thrive. Lyft’s Data Science Team builds mathematical models underpinning the platform’s core... ...and ML models to enable better decisions, and developing and evaluating algorithmic policies in our pricing, dispatch, and incentives...
Hourly pay
Summer work
Internship
Summer internship
Work at office
3 days per week
Lyft
Toronto, ON
8 days ago
DevOps Engineer - AI Model Evaluator
$85 per hour
...Mercor connects elite creative and technical talent with leading AI research labs. Headquartered in San Francisco, our investors... ...Responsibilities Use frontier AI coding agents to complete and evaluate complex infrastructure engineering tasks. Review model-...
Full time
Contract work
Summer work
Remote work
Mercor
Toronto, ON
more than 2 months ago
Data Science Manager
$80 - $100 per hour
...Mercor connects elite creative and technical talent with leading AI research labs. Headquartered in San Francisco, our investors... ...Angelo , Larry Summers , and Jack Dorsey . Position: Data Science Documents Type: Contract Compensation: $80–$100...
Contract work
Summer work
Remote work
Mercor
Toronto, ON
more than 2 months ago
Senior Data Engineer, AI & Agents
.... Endless possibilities. As a Senior Data Engineer, AI & Agents, you will prepare enterprise domain... ...stakeholders and subject-matter experts. The role centers on building and registering... ...’s or Master’s degree in Computer Science, Information Systems, Engineering, or a...
Full time
Contract work
Appnovation Technologies
Toronto, ON
16 days ago
Data Science & Strategy Analyst - Summer Internship in Canada
$6000 per month
...Job Description The Data Science & Strategy Team at Birch Hill is currently recruiting Summer Analysts for a four-month internship beginning... ...data-driven decision making through advanced analytics and AI. We lead a wide array of projects that support key components of...
Summer work
Internship
Summer internship
Birch Hill Inc
Toronto, ON
11 days ago
Vice President, Data Analytics & AI
...managers and practiced subject matter experts. And in each area of our evolving business... ...Officer (CTIO), the Vice President, Data Analytics & AI is the executive leader responsible for... ...or postgraduate education in computer science, data science, engineering, mathematics...
Long term contract
Full time
Internship
Work at office
Mattamy Homes
Toronto, ON
a month ago
AI Data Engineer
...Description Senior AI Data Engineer – Enterprise Data Build It. Scale It. Own It. Sangoma is a global leader in business communications, delivering cloud-based solutions that help organizations connect, collaborate, and engage with their customers. Our portfolio...
Long term contract
Full time
Work from home
Worldwide
Flexible hours
Sangoma
Toronto, ON
more than 2 months ago

Do you want to receive more vacancies?

Subscribe and receive similar vacancies to Data Science Expert - AI Evaluation. Be the first to apply!