Data Science Expert - AI Evaluation
$120 - $170 per hourMercor
Toronto, ON
- Remote job
About the job
Mercor connects elite creative and technical talent with leading AI research labs. Headquartered in San Francisco, our investors include Benchmark , General Catalyst , Peter Thiel , Adam D'Angelo , Larry Summers , and Jack Dorsey .
Position: Data Science Expert
Type: Contract
Compensation: $120–$170/hour
Location: Remote
Role Responsibilities
- Design precise, task-specific grading criteria for real-world data science deliverables such as analyses, models, dashboards, and experiment readouts.
- Score AI-generated and human work samples against criteria with detailed written justifications for every score.
- Apply consistent, evidence-based judgment to ensure scores are reproducible and defensible.
- Incorporate structured feedback from senior reviewers and iterate quickly on your work.
- Work independently and asynchronously to meet deadlines while improving AI model performance .
Qualifications
Must-Have
- 5+ years of professional data science experience in industry.
- Background in business operations, product, or growth data science at top-tier technology companies.
- Deep fluency in experiment design and A/B testing, metric definition, SQL/Python analysis, and communicating findings to executive stakeholders.
- Exceptionally strong written communication.
- Detail-oriented, consistent, and comfortable having your judgment reviewed and calibrated against peers.
Preferred
- Prior experience with AI training , evaluation, or human-data projects.
Application Process (Takes 20–30 mins to complete)
- Submit your resume or relevant technical background to get started.
- Qualified applicants may be asked to complete a brief technical assessment or submit additional information.
Resources & Support
- For details about the interview process and platform information, please check:
- For any help or support, reach out to: Show email
PS: Our team reviews applications daily. Please complete your AI interview and application steps to be considered for this opportunity.
Vacancy posted 1 day ago
Similar jobs that could be interesting for youBased on the Data Science Expert - AI Evaluation in Toronto, ON vacancy
$80 - $120 per hour
...Mercor connects elite creative and technical talent with leading AI research labs. Headquartered in San Francisco, our investors... ...Summers , and Jack Dorsey . Position: Software / AI / IT / data Evaluator Type: Contract Compensation: $80–$120/hour...SuggestedRemote jobContract workSummer workWork at office$80 - $120 per hour
...creative and technical talent with leading AI research labs. Headquartered in San... .... Position: Healthcare / clinical Evaluator Type: Contract Compensation... ...and asynchronously to ensure high-quality evaluations. Utilize Microsoft Office and Google...SuggestedRemote jobContract workSummer workWork at office$80 - $120 per hour
...Mercor connects elite creative and technical talent with leading AI research labs. Headquartered in San Francisco, our investors... ...Summers , and Jack Dorsey . Position: People ops / recruiting Evaluator Type: Contract Compensation: $80–$120/hour...SuggestedRemote jobContract workSummer workWork at office$80 - $150 per hour
...technical talent with leading AI research labs. Headquartered in... ...Position: UI / UX Design Expert Type: Contract... ...independently and asynchronously to evaluate and improve AI model performance... ...training , evaluation, or human-data projects. Application Process...SuggestedRemote jobContract workSummer work$80k - $180k per year
...Zafin is an AI platform company helping regulated institutions modernize... ...platform that connects data, systems, and workflows across... ...What’s the Opportunity? The AI Evaluation Engineer ensures AI agent... ...levels. Degree in Computer Science, Software Engineering, Data Science...SuggestedFull time$95k - $120k per year
...financial backgrounds. The Role: Credit Risk Expert & Strategic Builder We are looking for a hands-on, data-obsessed Data Science Manager, Risk to build and operationalize... ...: Partner with the Credit Risk team to evaluate underwriting policies, optimize risk segmentation...Full timeTemporary workInternshipWork at officeRemote work1 day per week- ...Challenge Ahead The Specialist Data Scientist applies advanced analytics, machine learning, and AI techniques to help solve... ...a member of Exploration's Data Science team, this role partners closely... ...both data science and geoscience experts, and the ability to translate complex...Temporary workWork at officeRemote workWork from homeWorldwideRelocationRelocation packageMonday to fridayFlexible hours
- ...and modernization, with a focus on stronger data visibility, better decision-making,... ...operational discipline, and practical adoption of AI across the business. This is a hybrid... ...Role We are creating a new Head of Data Science & Analytics role to build and lead the data...Full timeWork at office
$68.55 - $82.26 per hour
...RQ00745 -Requisition TitleData Science Developer Location: Toronto... ...that turns public health data into timely analytics, surveillance... ...prioritize use cases and integrate AI techniques (machine learning,... .... The client will evaluate candidates based on both rate expectations...- We are seeking a versatile Senior Data Science Developer to build, scale, and automate an enhanced... ...with advanced data science and ML/AI modeling. Modern Azure Stack: Work hands... ...Needs Assessment: Demonstrated background evaluating technical needs and resolving complex, multi...Long term contractContract work2 days per week
$80 - $120 per hour
...creative and technical talent with leading AI research labs. Headquartered in San... ...Position: Biology / environmental science Evaluator Type: Contract Compensation... ...with AI research teams to enhance training data quality and performance. Work independently...Remote jobContract workSummer workWork at office$80 - $120 per hour
...Mercor connects elite creative and technical talent with leading AI research labs. Headquartered in San Francisco, our investors... ...Dorsey . Position: Market research / competitive intelligence Evaluator Type: Contract Compensation: $80–$120/hour...Remote jobContract workSummer workWork at officeFlexible hours$80 per hour
...creative and technical talent with leading AI research labs. Headquartered in San Francisco... ..., and Jack Dorsey . Position: Data Engineer (Coding Agent Experience) Type... ...frontier AI coding agents to complete and evaluate complex data engineering tasks. Review...Remote jobContract workSummer work$50 per hour
...technical talent with leading AI research labs. Headquartered in... ...Dorsey . Position: Applied Sciences — Visual Document Understanding... ...Role Responsibilities Evaluate the accuracy and depth of AI-... ...research teams to improve training data quality and downstream...Remote jobContract workSummer work$80 - $120 per hour
...creative and technical talent with leading AI research labs. Headquartered in San... ...Position: Training / onboarding / L&D Evaluator Type: Contract Compensation... ...feedback. Collaborate with subject matter experts to ensure consistency and quality. Work...Remote jobContract workSummer workWork at office- ...Reference no. R2862231 Position title: Data and AI Engineer Department: Data... ...by one purpose: we chase the miracles of science to improve people’s lives. Our team, across... ..., and activities related to performance evaluation. Information collected and processed by...Full timeWork from homeFlexible hours
- ...Reporting to the Manager, Evaluation and Quality Improvement, the Program... ...#Master's degree in Health Sciences, Applied Social Sciences, Health... ...statistical packages used for data analysis in evaluation.... ...senior management and technical experts. ~Credentialed Evaluator designation...Hourly payFull timeTemporary workPart timeInternshipMonday to friday
$70 per hour
...creative and technical talent with leading AI research labs. Headquartered in San... ...Jack Dorsey . Position: Generalist Expert Type: Contract Compensation... ...Remote Role Responsibilities Evaluate AI-generated responses to ensure accuracy...Remote jobContract workSummer work$110k - $160k per year
...working world for all. As AI Engineer, you will design and build... ..., multi-agent coordination, evaluation frameworks, observability and... ...Architect, AI Platform Engineers, and Data Scientists to translate complex... ...have ~ A degree in Computer Science, Data Science, Engineering or...Long term contract- ...curriculum, developed in partnership with Nebius AI. Our team is fully remote and globally... ...of expertise We will bring on three Experts , each focused on one of the following areas... ...areas. Expert A — System Foundations, API & Data Module 1 — Foundations of System...Work at officeRemote workFlexible hours
$29 - $45 per hour
...creative and technical talent with leading AI research labs. Headquartered in San Francisco... ...Jack Dorsey . Position: AI Safety Experts — English & Portuguese (global) Type:... ...exploitation. Generate high-quality human data by annotating failures, classifying...Remote jobContract workSummer work$85k per year
...The University of Toronto is pleased to announce the third call for applications for the Eric and Wendy Schmidt AI in Science Postdoctoral Fellowship , a program of Schmidt Sciences, which brings together the brightest minds who have completed a PhD in the natural sciences...Full time$68k - $72k per year
...fill a new position of Manager, Planning and Evaluation. This is a full-time position working 35... ..., including workflow development, data tracking, analysis, and reporting. ~... ...As part of our recruitment process, we use AI-enabled tools to assist with summarizing applicant...Full timeInternshipWork at officeFlexible hoursWeekend work$135k - $210k per year
...Overview: Guidepoint seeks an experienced Data/AI Engineer as an integral member of the... ...Best Practices: Collaborate with data science and engineering teams to establish and implement... ...for LLMOps, including automated evaluation using frameworks like LLM Judges or MLflow...Full timeWorldwide- ...Analytics is looking for experienced Data Scientists GenAI to join our fast-growing... ...bring deep expertise in Data Science, Machine Learning and AI. We are the trusted analytics partner... ...JumpStart, HuggingFace ~ Experience evaluating ethical implications of AI and controlling...Full timeLocal area
$40 per hour
...purchase journey by combining technology, data, and creativity with commercial... ...boost your personal development in data science, analytics, and AI? Looking for an innovative, agile work... ...internal and external business and technology experts from around the world. Work closely...InternshipSummer internshipWork at officeWork from homeFlexible hours- ...is seeking a highly motivated Research Coordinator with excellent data analysis skills to assist the team with active clinical research... ...Qualifications/skills Requires the successful completion of BSc. in a science related discipline or equivalent research experience A minimum...Full timeTemporary workInternship
$85 per hour
...Mercor connects elite creative and technical talent with leading AI research labs. Headquartered in San Francisco, our investors... ...Responsibilities Use frontier AI coding agents to complete and evaluate complex infrastructure engineering tasks. Review model-...Remote jobContract workSummer work$122k - $192k per year
...looking for an experienced DSM Evaluation Manager to join our expanding... ...staff, coordinating fieldwork, data collection, energy modeling,... ...e. engineering, environmental science with a focus on energy efficiency... ...efficiency programs. ~ Expert level understanding of building...Full timeInternshipWork at officeLocal area$60 - $75 per hour
...Mercor connects elite creative and technical talent with leading AI research labs. Headquartered in San Francisco, our investors... ...Larry Summers , and Jack Dorsey . Position: Software & Data Science Expert Type: Contract Compensation: $60–$75/hour...Remote jobContract workSummer work
Do you want to receive more vacancies?
Subscribe and receive similar vacancies to Data Science Expert - AI Evaluation. Be the first to apply!
Related searches