Data Scientist - AI Evaluation

$100 - $150 per hour

Mercor

Remote job

About the job

Mercor connects elite creative and technical talent with leading AI research labs. Headquartered in San Francisco, our investors include Benchmark , General Catalyst , Peter Thiel , Adam D'Angelo , Larry Summers , and Jack Dorsey .

Position: Data Scientist Talent Network

Type: Contract

Compensation: $100–$150/hour

Location: Remote

Role Responsibilities

Design precise, task-specific grading criteria for data science deliverables. This includes exploratory data analyses , statistical modeling work , machine learning pipelines , and technical reports.
Evaluate AI-generated or human-created work against established criteria to ensure quality and accuracy.
Provide detailed written justifications for evaluations and scores to maintain transparency.
Apply consistent, evidence-based judgment for reproducible and defensible assessments.
Incorporate structured feedback from senior reviewers and iterate on submitted work for continuous improvement.

Qualifications

Must-Have

1+ years of professional data science experience.
Experience at a leading technology, research, or quantitative firm.
Strong command of Python , SQL , statistical modeling , and machine learning .
Exceptional written communication skills for conveying technical findings.
Detail-oriented and consistent approach to evaluating complex work.

Application Process (Takes 20–30 mins to complete)

Upload resume
AI interview based on your resume
Submit form

Resources & Support

For details about the interview process and platform information, please check:
For any help or support, reach out to: Show email

PS: Our team reviews applications daily. Please complete your AI interview and application steps to be considered for this opportunity.

Vacancy posted 2 days ago

Similar jobs that could be interesting for youBased on the Data Scientist - AI Evaluation in Toronto, ON vacancy

Data Scientist - AI Evaluation
$100 - $150 per hour
...creative and technical talent with leading AI research labs. Headquartered in San... ...Summers , and Jack Dorsey . Position: Data Scientist Talent Network Type: Contract... ...pipelines , and A/B test write-ups . Evaluate AI-generated or human-created work...
Suggested
Full time
Contract work
Summer work
Remote work
Mercor
Toronto, ON
more than 2 months ago
Data Engineer - AI Model Evaluator
$80 per hour
...creative and technical talent with leading AI research labs. Headquartered in San Francisco... ..., and Jack Dorsey . Position: Data Engineer (Coding Agent Experience) Type... ...frontier AI coding agents to complete and evaluate complex data engineering tasks. Review...
Suggested
Full time
Contract work
Summer work
Remote work
Mercor
Toronto, ON
more than 2 months ago
Sr. Data Scientist
...you love? It’s Possible. At Pinterest, AI isn't just a feature, it's a powerful... ...here . We are looking for a Senior Data Scientist. You will shape the future of people-facing... ...outputs. ~ Strong track record of critical evaluation and verification of AI-assisted work (e.g...
Suggested
Full time
Work at office
Relocation
Relocation package
Pinterest
Toronto, ON
21 days ago
Data Scientist
...Position: Data Scientist Location: Toronto, ON Job ID#: RQ11452 Duration: 12 Months Role Overview We are seeking... ...them staff difficult assignments with proven Experts uniquely evaluated by our proven QMS- Quality Management System Staff Development...
Suggested
Full time
Symbiotic Digital
Toronto, ON
27 days ago
Data Scientist
$68.8k - $100k per year
...12-month contract opportunity* The Data Scientist will be a pivotal member of the Enterprise... ...and implementing advanced analytics and AI models. Leverage statistical techniques,... ...Critical Thinking – The ability to analyze, evaluate and interpret information from various...
Suggested
Full time
Contract work
Work at office
Remote work
Meridian Credit Union
Toronto, ON
more than 2 months ago
Data Scientist
$80.7k - $100k per year
..., enforcement, and education. We help build a safer community. Data analytics sits at the heart of TSSA’s vision for a safer Ontario.... ...with us. Your Key Contributions: Identify opportunities for AI and advanced analytics , applying data science techniques to...
Full time
T S S A Inc
Toronto, ON
more than 2 months ago
Data Scientist
...strong, globally distributed team from the start. We need a Data Scientist to help design and build the algorithms that power insights, pre... ...trends, and opportunities for product value. Continuously evaluate model performance and iterate to improve accuracy, reliability...
Full time
Part time
Reflow
Toronto, ON
more than 2 months ago
Data Scientist
$80 - $100 per hour
...Mercor connects elite creative and technical talent with leading AI research labs. Headquartered in San Francisco, our investors include... ...D'Angelo , Larry Summers , and Jack Dorsey . Position: Data Science Documents Type: Contract Compensation:...
Full time
Contract work
Summer work
Remote work
Mercor
Toronto, ON
more than 2 months ago
Data Scientist, Decisions - Mapping
$88k - $110k per year
...We aim to achieve this by cultivating a work environment where all team members belong and have the opportunity to thrive. As a Data Scientist on the Mapping team, you will collaborate with our world class team of engineers, product managers, and designers to grow and...
Hourly pay
Full time
Work at office
Flexible hours
3 days per week
Lyft
Toronto, ON
21 days ago
Senior Data Scientist
$120k - $160k per year
...trusted platforms for financial data access, enrichment, and money... ...We're hiring a Senior Data Scientist to own machine-learning models... ...retraining, champion-challenger evaluation and promotion. Partner... ...and rollout. Move fast with AI-assisted development -we use...
Full time
Contract work
Apprenticeship
Remote work
Flexible hours
Flinks
Toronto, ON
more than 2 months ago
Data Scientist, Risk Modeling & Analytics
$75k - $100k per year
...About the role: Do you love diving into data, solving complex problems with tech, and... ...to hear from you! We are seeking a Data Scientist, Risk Analytics & Modeling, to work under... ...automation, data pipelines, machine learning, and AI to your work Participate in strategic...
Full time
Internship
Work at office
Flexible hours
Financeit
Toronto, ON
19 days ago
Senior Data Scientist, Financial Services
$150k - $200k per year
...Senior Data Scientist, Financial Services About Clutch: Clutch is Canada's largest online used car retailer, delivering a seamless, hassle-free car-buying experience to drivers everywhere. Customers can browse hundreds of cars from the comfort of their home, get the right...
Permanent employment
Full time
Internship
Immediate start
Clutch Technologies
Toronto, ON
14 days ago
Sr. Data Scientist, Business Analytics
$168.3k - $217.8k per year
...your home — from personalized guidance to AI tools and a best-in-class hiring experience... ...strategy and team execution, and we use data to tell Thumbtack’s story to our team, our... ...impact. About the role As a Sr. Data Scientist on Business Analytics, you’ll drive...
Full time
Seasonal work
Local area
Thumbtack
Toronto, ON
27 days ago
Data Scientist
...Company Description You will join a world-class team of engineers and data scientists from Facebook, Uber, Amazon and Google. We are a fast growing consulting firm based in Toronto with clients ranging from leading startups building impactful technologies to Fortune 500...
Full time
Ample Insight
Toronto, ON
more than 2 months ago
Senior Data Scientist
$100k per year
...About the Job: \ We are seeking a business -focused Data Scientist \ to use data science, advanced analytics, Machine Learning, and AI to generate insights, improve customer experience, optimize operations, and support strategic decision -making across business...
Permanent employment
Full time
Work at office
Local area
Infoya
Toronto, ON
more than 2 months ago
Data Scientist - ML Specialist
$60 - $100 per hour
...creative and technical talent with leading AI research labs. Headquartered in San... ...Jack Dorsey . Position: Exceptional Data Scientists Type: Contract Compensation... ...to build predictive models. Evaluate and improve AI models using TensorFlow...
Full time
Contract work
Summer work
Remote work
Mercor
Toronto, ON
more than 2 months ago
Data Analyst / Data Scientist
...Job role: As a data analyst, you will be responsible for compiling actionable insights from data and assisting program, sales and marketing managers build data-driven processes. Your role will involve driving initiatives to optimize for operational excellence and revenue...
Full time
One 37 Inc
Toronto, ON
more than 2 months ago
Compliance AI Evaluator - Finance
$80 - $120 per hour
...Mercor connects elite creative and technical talent with leading AI research labs. Headquartered in San Francisco, our investors... ...Position: Compliance / regulatory response with financial-services AI Evaluator Type: Contract Compensation: $80–$120/hour...
Full time
Contract work
Summer work
Work at office
Remote work
Mercor
Toronto, ON
more than 2 months ago
RQ00746-Data Analytical Specialist/Scientist
$73.38 - $88.06 per hour
...RQ00746-Data Analytical Specialist/Scientist Location: 661 University Ave, Suite 1701, Toronto, ON M5G 1M1... ...sustainable knowledge transfer. AI Disclaimer: Source Code may use artificial... ...have qualifications. The client will evaluate candidates based on both rate...
Full time
Contract work
Work at office
Source Code
Toronto, ON
more than 2 months ago
Financial Compliance AI Evaluator
$80 - $120 per hour
...Mercor connects elite creative and technical talent with leading AI research labs. Headquartered in San Francisco, our investors... ...Position: Compliance / regulatory response with financial-services AI Evaluator Type: Contract Compensation: $80–$120/hour...
Full time
Contract work
Summer work
Work at office
Remote work
Mercor
Toronto, ON
more than 2 months ago
Data Scientist - Data Analytics and Infrastructure
...Company Description You will join a world-class team of engineers and data scientists from Facebook, Uber, Amazon and Google. We are a fast growing consulting firm based in Toronto with clients ranging from leading startups building impactful technologies to Fortune 500...
Full time
Ample Insight
Toronto, ON
more than 2 months ago
Data Scientist - Algorithms, Mapping
$108k - $135k per year
...We aim to achieve this by cultivating a work environment where all team members belong and have the opportunity to thrive. As a Data Scientist on the Mapping team, you will collaborate with our world class team of scientists, engineers, product managers, and designers to...
Hourly pay
Work at office
Flexible hours
3 days per week
Lyft
Toronto, ON
a month ago
Data Scientist, Decisions - Verticals
$108k - $135k per year
...where all team members belong and have the opportunity to thrive. Data Science is at the heart of Lyft’s products and decision-making.... ...a seamless experience for both riders and drivers. As a Data Scientist on the Verticals team, you will collaborate with engineering,...
Hourly pay
Full time
Work at office
Flexible hours
3 days per week
Lyft
Toronto, ON
more than 2 months ago
Senior Data Scientist - F500 Expert
$60 - $70 per hour
...creative and technical talent with leading AI research labs. Headquartered in San... ...Summers , and Jack Dorsey . Position: Data Science and Analytics Experts Type:... ...-Have ~5+ years working as a data scientist, analytics leader, or ML engineer at a Fortune...
Full time
Contract work
Summer work
Remote work
Mercor
Toronto, ON
more than 2 months ago
DevOps Engineer - AI Model Evaluator
$85 per hour
...Mercor connects elite creative and technical talent with leading AI research labs. Headquartered in San Francisco, our investors... ...Responsibilities Use frontier AI coding agents to complete and evaluate complex infrastructure engineering tasks. Review model-...
Full time
Contract work
Summer work
Remote work
Mercor
Toronto, ON
more than 2 months ago
RQ11420 - Data Analytical Specialist/Scientist - Senior
...Government of Ontario is embarking on a large Data & AI transformation initiative. Core to the... ...the future. We are looking for a data scientist to support various initiatives related... ...escalate model anomalies. ~ Pipeline & Evaluation Support: Assist with running DSPy...
Full time
Maarut Inc
Toronto, ON
a month ago
Data Scientist - Applied AI/Gen AI
...Reference No. R2873670 Position Title: Data Scientist Location: Toronto, ON - Hybrid - 3... ...About Sanofi We’re an R&D-driven, AI-powered biopharma company committed to improving... ..., and activities related to performance evaluation. Information collected and processed by the...
Work from home
3 days per week
Toronto, ON
1 day ago
Senior Data Engineer, AI Systems
$144k - $188k per year
...personalization for marketers through data-activated content generation and AI decisioning. The world’s most... ...will work alongside ML engineers and scientists in a collaborative environment, contributing... ...model development, training, and evaluation Identify and resolve performance...
Full time
Movable Ink
Toronto, ON
more than 2 months ago
AI COE - Senior Data Scientist
$88k - $132k per year
...Job Summary As a Senior Data Scientist, you will be an integral part of the AI Centre of Excellence (COE) within the Office of the Chief Technology Officer... ...Application Engineering: Implement, fine-tune, and evaluate large language model (LLM) solutions for diverse use...
Work at office
Immediate start
Weekend work
Ernst & Young
Toronto, ON
23 days ago
Data Scientist, ONCAP/OpCo - 12 month contract
$45 - $60 per hour
...The Opportunity: ONCAP is seeking a Data Scientist to support the AI, Analytics, and Data function within one of its portfolio companies. The selected... ...AI and data-driven targeting techniques to identify, evaluate, and prioritize high-potential acquisition targets aligned...
Full time
Contract work
Local area
Shift work
Oncap
Toronto, ON
more than 2 months ago

Do you want to receive more vacancies?

Subscribe and receive similar vacancies to Data Scientist - AI Evaluation. Be the first to apply!