Data Science Expert - AI Evaluation

$120 - $170 per hour

Full-time

Mercor

About the job

Mercor connects elite creative and technical talent with leading AI research labs. Headquartered in San Francisco, our investors include Benchmark , General Catalyst , Peter Thiel , Adam D'Angelo , Larry Summers , and Jack Dorsey .

Position: Data Science Expert

Type: Contract

Compensation: $120–$170/hour

Location: Remote

Role Responsibilities

Design precise, task-specific grading criteria for real-world data science deliverables such as analyses, models, dashboards, and experiment readouts.
Score AI-generated and human work samples against criteria with detailed written justifications for every score.
Apply consistent, evidence-based judgment to ensure scores are reproducible and defensible.
Incorporate structured feedback from senior reviewers and iterate quickly on your work.
Work independently and asynchronously to meet deadlines while improving AI model performance .

Qualifications

Must-Have

5+ years of professional data science experience in industry.
Background in business operations, product, or growth data science at top-tier technology companies.
Deep fluency in experiment design and A/B testing, metric definition, SQL/Python analysis, and communicating findings to executive stakeholders.
Exceptionally strong written communication.
Detail-oriented, consistent, and comfortable having your judgment reviewed and calibrated against peers.

Preferred

Prior experience with AI training , evaluation, or human-data projects.

Application Process (Takes 20–30 mins to complete)

Submit your resume or relevant technical background to get started.
Qualified applicants may be asked to complete a brief technical assessment or submit additional information.

Resources & Support

For details about the interview process and platform information, please check:
For any help or support, reach out to: View email address on jobs.jobcopilot.com

PS: Our team reviews applications daily. Please complete your AI interview and application steps to be considered for this opportunity.

Apply

Vacancy posted 2 hours ago

Similar jobs that could be interesting for youBased on the Data Science Expert - AI Evaluation in Toronto, ON vacancy

Data Science Expert - Evaluation Specialist
$120 - $170 per hour
...Mercor connects elite creative and technical talent with leading AI research labs. Headquartered in San Francisco, our investors... ...Angelo , Larry Summers , and Jack Dorsey . Position: Data Science Expert Type: Contract Compensation: $120–$170/hour...
Suggested
Full time
Contract work
Summer work
Remote work
Mercor
Toronto, ON
2 hours ago
Data Quality Expert - Evaluator
$80 - $120 per hour
...connects elite creative and technical talent with leading AI research labs. Headquartered in San Francisco, our investors... ..., Larry Summers , and Jack Dorsey . Position: Data quality / CRM operations Evaluator Type: Contract Compensation: $80–$120/hour...
Suggested
Full time
Contract work
Summer work
Work at office
Remote work
Mercor
Toronto, ON
2 hours ago
Data Scientist - AI Evaluation
$100 - $150 per hour
...technical talent with leading AI research labs. Headquartered in... ...Jack Dorsey . Position: Data Scientist Talent Network... ...specific grading criteria for data science deliverables, including exploratory... ...A/B test write-ups . Evaluate AI-generated or human-created...
Suggested
Full time
Contract work
Summer work
Remote work
Mercor
Toronto, ON
10 hours ago
Data Engineer - AI Model Evaluation
$80 per hour
...creative and technical talent with leading AI research labs. Headquartered in San Francisco... ..., and Jack Dorsey . Position: Data Engineer (Coding Agent Experience) Type... ...frontier AI coding agents to complete and evaluate complex data engineering tasks. Review...
Suggested
Full time
Contract work
Summer work
Remote work
Mercor
Toronto, ON
10 hours ago
Privacy Compliance Evaluator - Expert
$80 - $120 per hour
...creative and technical talent with leading AI research labs. Headquartered in San... ...Position: Privacy / regulatory compliance Evaluator Type: Contract Compensation... ...with AI research teams to enhance training data quality. Work independently and asynchronously...
Suggested
Full time
Contract work
Summer work
Work at office
Remote work
Flexible hours
Mercor
Toronto, ON
2 hours ago
Sales Engineering Expert - Evaluator
$100 - $150 per hour
...technical talent with leading AI research labs. Headquartered in... ...Position: Sales Engineering Expert Type: Contract... ...supporting proofs of concept, and evaluating product fit and technical risk.... ...infrastructure, cybersecurity, data platforms, developer tools, or...
Full time
Contract work
Summer work
Remote work
Mercor
Toronto, ON
10 hours ago
Privacy Compliance Expert - Evaluator
$80 - $120 per hour
...Mercor connects elite creative and technical talent with leading AI research labs. Headquartered in San Francisco, our investors... ...Jack Dorsey . Position: Privacy / regulatory compliance Evaluator Type: Contract Compensation: $80–$120/hour...
Full time
Contract work
Summer work
Work at office
Remote work
Mercor
Toronto, ON
2 hours ago
Customer Success Evaluator - Expert
$80 - $120 per hour
...Mercor connects elite creative and technical talent with leading AI research labs. Headquartered in San Francisco, our investors... ...Dorsey . Position: Customer success / support operations Evaluator Type: Contract Compensation: $80–$120/hour...
Full time
Contract work
Summer work
Work at office
Remote work
Mercor
Toronto, ON
10 hours ago
Business Operations Expert - Evaluator
$60 - $80 per hour
...connects elite creative and technical talent with leading AI research labs. Headquartered in San Francisco, our investors... ...Summers , and Jack Dorsey . Position: Business Domain Expert — AI Model Evaluation Pilot (Admin, Marketing, HR, Accounting) Type:...
Hourly pay
Full time
Contract work
For contractors
Summer work
Remote work
Mercor
Toronto, ON
10 hours ago
Legal Compliance Expert - Evaluator
$80 - $120 per hour
...Mercor connects elite creative and technical talent with leading AI research labs. Headquartered in San Francisco, our investors... ...Summers , and Jack Dorsey . Position: Legal / compliance Evaluator Type: Contract Compensation: $80–$120/hour...
Full time
Contract work
Summer work
Work at office
Remote work
Mercor
Toronto, ON
2 hours ago
Software Engineer - Data Science
$90 - $110 per hour
...Mercor connects elite creative and technical talent with leading AI research labs. Headquartered in San Francisco, our investors... ...Larry Summers , and Jack Dorsey . Position: Software & Data Science Expert Type: Contract Compensation: $90–$110/hour...
Full time
Contract work
Summer work
Remote work
Mercor
Toronto, ON
10 hours ago
AI Evaluation Engineer (QA)
...digital experiences and client partnerships. As a QA / AI Evaluation Engineer, you will join a highly motivated and... ...4+ years in QA / test engineering, with exposure to data/ML systems. ~ Strong Python and data-science techniques for measuring factual grounding and answer...
Full time
Immediate start
Appnovation Technologies
Toronto, ON
2 hours ago
Senior Data Science Analyst
$78.95k - $118.42k per year
...We seek a talented and driven Data Scientist passionate about accelerating... ...through machine learning and AI to join our team at the... ...contribute to our mission of open science and pre-competitive drug discovery... ...or in collaboration with other experts in the Haibe-Kains and Schapira...
Remplacement
Permanent employment
Full time
Internship
Manual labor
Flexible hours
Shift work
Day shift
University Health Network
Toronto, ON
10 hours ago
RQ00745-Data Science Developer
$68.55 - $82.26 per hour
...RQ00745 -Requisition TitleData Science Developer Location: Toronto... ...that turns public health data into timely analytics, surveillance... ...prioritize use cases and integrate AI techniques (machine learning,... .... The client will evaluate candidates based on both rate expectations...
Full time
Source Code
Toronto, ON
10 hours ago
Data Science Manager
$80 - $100 per hour
...Mercor connects elite creative and technical talent with leading AI research labs. Headquartered in San Francisco, our investors... ...Angelo , Larry Summers , and Jack Dorsey . Position: Data Science Documents Type: Contract Compensation: $80–$100...
Full time
Contract work
Summer work
Remote work
Mercor
Toronto, ON
2 hours ago
Enterprise Data Science Specialist
$60 - $70 per hour
...connects elite creative and technical talent with leading AI research labs. Headquartered in San Francisco, our investors... ...Angelo , Larry Summers , and Jack Dorsey . Position: Data Science and Analytics Experts Type: Contract Compensation: $60–$70/...
Full time
Contract work
Summer work
Remote work
Mercor
Toronto, ON
10 hours ago
Manager, Data Science & Machine Learning
$155k - $165k per year
...right place! We’re looking for a Manager, Data Science & Machine Learning to join our Data team... ...failure are encouraged. This includes new AI/ML features or other experimental... ...stakeholders and other Data Office leads as an expert to help design and translate business problems...
Full time
Work at office
Remote work
Flexible hours
Lightspeed Commerce
Toronto, ON
2 hours ago
Manager, Data Science
...and solve challenging problems and make significant and lasting impact on Canadians. We are seeking an experienced Manager of Data Science & Engineering to lead our Budget Optimization, Forecasting, and Analytics initiatives in our Media & Advertising domain. There would...
Permanent employment
Full time
Temporary work
Internship
Loblaw Digital
Toronto, ON
2 hours ago
Financial Compliance AI Evaluator
$80 - $120 per hour
...Mercor connects elite creative and technical talent with leading AI research labs. Headquartered in San Francisco, our investors... ...Position: Compliance / regulatory response with financial-services AI Evaluator Type: Contract Compensation: $80–$120/hour...
Full time
Contract work
Summer work
Work at office
Remote work
Mercor
Toronto, ON
2 hours ago
Head of Data Science & Analytics
...and modernization, with a focus on stronger data visibility, better decision-making,... ...operational discipline, and practical adoption of AI across the business. This is a hybrid... ...Role We are creating a new Head of Data Science & Analytics role to build and lead the data...
Full time
Work at office
Big Viking Games
Toronto, ON
2 hours ago
DevOps Engineer - AI Model Evaluator
$85 per hour
...Mercor connects elite creative and technical talent with leading AI research labs. Headquartered in San Francisco, our investors... ...Responsibilities Use frontier AI coding agents to complete and evaluate complex infrastructure engineering tasks. Review model-...
Full time
Contract work
Summer work
Remote work
Mercor
Toronto, ON
10 hours ago
Data Science Manager, Rider App
$172k - $215k per year
...all team members belong and have the opportunity to thrive. Data Science is at the heart of Lyft’s products and decision-making. Data Scientists... ...emerging research, technologies, and industry best practices in AI powered data science workflow. Experience: ~ Master’s...
Hourly pay
Long term contract
Full time
Temporary work
Work at office
Flexible hours
3 days per week
Lyft
Toronto, ON
10 hours ago
Senior Data/AI Engineer
$135k - $210k per year
...Overview: Guidepoint seeks an experienced Data/AI Engineer as an integral member of the... ...Best Practices: Collaborate with data science and engineering teams to establish and implement... ...for LLMOps, including automated evaluation using frameworks like LLM Judges or MLflow...
Full time
Worldwide
Guidepoint
Toronto, ON
10 hours ago
Senior Data Scientist - F500 Expert
$60 - $70 per hour
...connects elite creative and technical talent with leading AI research labs. Headquartered in San Francisco, our investors... ...Angelo , Larry Summers , and Jack Dorsey . Position: Data Science and Analytics Experts Type: Contract Compensation: $60–$70/...
Full time
Contract work
Summer work
Remote work
Mercor
Toronto, ON
10 hours ago
Marketing Operations Expert - AI Environments
$60 - $100 per hour
...technical talent with leading AI research labs. Headquartered in... ...Dorsey . Position: Marketing Expert — Marketing Operations (AI... ...wrong responses. Review and evaluate AI agent attempts against... ...Diagnose issues from messy data, prioritize tasks, plan, execute...
Full time
Contract work
Summer work
Remote work
Mercor
Toronto, ON
2 hours ago
IT Consulting Expert - AI/ML
$65 - $85 per hour
...and technical talent with leading AI research labs. Headquartered in San... ...Responsibilities Review and evaluate AI-generated outputs related to solution... ...requirements gathering, and vendor evaluation. Annotate, label, and validate data across IT services use cases like...
Full time
Contract work
Summer work
Remote work
Mercor
Toronto, ON
10 hours ago
AI Scenario Designer - Creative Expert
$30 - $80 per hour
...creative and technical talent with leading AI research labs. Headquartered in San Francisco... .... Run the agent against scenarios, evaluate performance, and guide it to success through... ...designer or UX researcher. Technical/data comfort with JSON , Jira/Asana , or exposure...
Full time
Contract work
Summer work
Remote work
Mercor
Toronto, ON
10 hours ago
Director, Data & AI (Hybrid)
...Starts Here We’re looking for a Director, Data & AI to lead our Data Engineering, Data & ML Platform, Data Science, and Applied AI teams. You'll own the strategy,... ...experimentation frameworks, guardrail metrics, and evaluation pipelines that drive confident product...
Hourly pay
Long term contract
Full time
Temporary work
Internship
Work at office
Local area
Work from home
Relocation
Flexible hours
Homebase
Toronto, ON
2 hours ago
Manager, Data & AI Strategy
...Consulting, Digital Marketing Services, Data & AI, Strategy Consulting, Business Operations... ...aligned to business objectives Define evaluation frameworks (value, feasibility, data readiness... ...Bachelor’s degree in Business, Computer Science, Engineering, Data Science, or related...
Remote job
Long term contract
Full time
Work at office
Telus Digital
Toronto, ON
2 hours ago
Clinical Data Manager - CDISC Expert
$100 - $150 per hour
...Mercor connects elite creative and technical talent with leading AI research labs. Headquartered in San Francisco, our investors include... ..., Larry Summers , and Jack Dorsey . Position: Clinical Data Manager / CDISC Type: Contract Compensation: $...
Full time
Contract work
Summer work
Remote work
Mercor
Toronto, ON
2 hours ago

Do you want to receive more vacancies?

Subscribe and receive similar vacancies to Data Science Expert - AI Evaluation. Be the first to apply!