Data Scientist - AI Evaluation
$100 - $150 per hourFull-time
Mercor
About the job
Mercor connects elite creative and technical talent with leading AI research labs. Headquartered in San Francisco, our investors include Benchmark , General Catalyst , Peter Thiel , Adam D'Angelo , Larry Summers , and Jack Dorsey .
Position: Data Scientist Talent Network
Type: Contract
Compensation: $100–$150/hour
Location: Remote
Role Responsibilities
- Design precise, task-specific grading criteria for data science deliverables, including exploratory data analyses , statistical modeling work , machine learning pipelines , and A/B test write-ups .
- Evaluate AI-generated or human-created work against established criteria to ensure quality and consistency.
- Provide detailed written justifications for evaluations and scores to maintain transparency and accountability.
- Apply consistent, evidence-based judgment to ensure assessments are reproducible and defensible.
- Incorporate structured feedback from senior reviewers and iterate on submitted work for continuous improvement.
- Work independently and asynchronously to meet deadlines and improve AI model performance .
Qualifications
Must-Have
- 1+ years of professional data science experience.
- Experience at a leading technology, research, or quantitative firm.
- Strong command of Python , SQL , statistical modeling , machine learning , and experimentation .
- Exceptional written communication skills.
- Detail-oriented and consistent approach to evaluating complex work.
- Comfort receiving feedback and calibrating judgment against established standards.
Application Process (Takes 20–30 mins to complete)
- Upload resume
- AI interview based on your resume
- Submit form
Resources & Support
- For details about the interview process and platform information, please check:
- For any help or support, reach out to: View email address on jobs.jobcopilot.com
PS: Our team reviews applications daily. Please complete your AI interview and application steps to be considered for this opportunity.
Vacancy posted 23 hours ago
Similar jobs that could be interesting for youBased on the Data Scientist - AI Evaluation in Toronto, ON vacancy
$100 - $150 per hour
...creative and technical talent with leading AI research labs. Headquartered in San... ...Summers , and Jack Dorsey . Position: Data Scientist Talent Network Type: Contract... ...pipelines , and technical reports. Evaluate AI-generated or human-created work against...SuggestedRemote jobContract workSummer work$120 - $170 per hour
...creative and technical talent with leading AI research labs. Headquartered in San... ...Summers , and Jack Dorsey . Position: Data Science Expert Type: Contract... ...~ Prior experience with AI training , evaluation, or human-data projects. Application...SuggestedFull timeContract workSummer workRemote work$80 per hour
...creative and technical talent with leading AI research labs. Headquartered in San Francisco... ..., and Jack Dorsey . Position: Data Engineer (Coding Agent Experience) Type... ...frontier AI coding agents to complete and evaluate complex data engineering tasks. Review...SuggestedFull timeContract workSummer workRemote work$96k - $125k per year
...focused problem solvers who thrive on complex data challenges. Guided by our values of Care,... ...-5 years of hands-on experience as a Data Scientist ~ Proficiency in Python and SQL, with... ...with LLMs, prompt engineering, and AI application development using APIs (e.g. OpenAI...SuggestedFull time$68.8k - $100k per year
...12-month contract opportunity* The Data Scientist will be a pivotal member of the Enterprise... ...and implementing advanced analytics and AI models. Leverage statistical techniques,... ...Critical Thinking – The ability to analyze, evaluate and interpret information from various...SuggestedFull timeContract workWork at officeRemote work$80 - $100 per hour
...Mercor connects elite creative and technical talent with leading AI research labs. Headquartered in San Francisco, our investors include... ...D'Angelo , Larry Summers , and Jack Dorsey . Position: Data Science Documents Type: Contract Compensation:...Full timeContract workSummer workRemote work$115k - $140k per year
...recruiting for a mission-driven AI company that is dedicated to... ...meaningful impact. Join the Data team! High velocity, high... ...They are seeking a Data Scientist to join the team. Do you love... ...AI Solutions: Prototyping and evaluating new AI-powered functions for job...Full timeWork at office$80.7k - $100k per year
..., enforcement, and education. We help build a safer community. Data analytics sits at the heart of TSSA’s vision for a safer Ontario.... ...with us. Your Key Contributions: Identify opportunities for AI and advanced analytics , applying data science techniques to...Full time$108k - $150k per year
...role, you'll research and develop innovative AI and Machine Learning based approaches to... ...to autonomously own part of our data strategy, work in one of the world's most... ...' experience. We're looking for a Data Scientist with a strong background in applied Machine...Permanent employmentFull timeWork at officeFlexible hours- ...strong, globally distributed team from the start. We need a Data Scientist to help design and build the algorithms that power insights, pre... ...trends, and opportunities for product value. Continuously evaluate model performance and iterate to improve accuracy, reliability...Full timePart time
$120k - $160k per year
...trusted platforms for financial data access, enrichment, and money... ...We're hiring a Senior Data Scientist to own machine-learning models... ...retraining, champion-challenger evaluation and promotion. Partner... ...and rollout. Move fast with AI-assisted development -we use...Contract workApprenticeshipRemote workFlexible hours$80 - $120 per hour
...connects elite creative and technical talent with leading AI research labs. Headquartered in San Francisco, our investors... ..., Larry Summers , and Jack Dorsey . Position: Data quality / CRM operations Evaluator Type: Contract Compensation: $80–$120/hour...Full timeContract workSummer workWork at officeRemote work- ...Company Description You will join a world-class team of engineers and data scientists from Facebook, Uber, Amazon and Google. We are a fast growing consulting firm based in Toronto with clients ranging from leading startups building impactful technologies to Fortune 500...Full time
- ...successful digital experiences and client partnerships. As a QA / AI Evaluation Engineer, you will join a highly motivated and experienced team... .... ~4+ years in QA / test engineering, with exposure to data/ML systems. ~ Strong Python and data-science techniques for measuring...Full timeImmediate start
$100k per year
...About the Job: \ We are seeking a business -focused Data Scientist \ to use data science, advanced analytics, Machine Learning, and AI to generate insights, improve customer experience, optimize operations, and support strategic decision -making across business...Permanent employmentFull timeWork at officeLocal area- ...we’re shaping the future of automotive retail in Canada and beyond. As Senior Data Scientist , you will be key contributor within our global Data Science organization, shaping AI and machine learning systems that sit at the core of our marketplace platforms across...Long term contractPermanent employmentFull time
$60 - $100 per hour
...creative and technical talent with leading AI research labs. Headquartered in San... ...Jack Dorsey . Position: Exceptional Data Scientists Type: Contract Compensation... ...to build predictive models. Evaluate and improve AI models using TensorFlow...Full timeContract workSummer workRemote work- ...Guideline’s proprietary spend and pricing data represents approximately $200 billion in annual... ...our investment in analytics and AI-powered solutions for the advertising and... ...revenue across multiple sales teams. The Data Scientist, Insights, will report to the Chief Insights...Full timeWork at office2 days per week
$80 - $120 per hour
...Mercor connects elite creative and technical talent with leading AI research labs. Headquartered in San Francisco, our investors... ...Position: Compliance / regulatory response with financial-services AI Evaluator Type: Contract Compensation: $80–$120/hour...Full timeContract workSummer workWork at officeRemote work- ...Principal Data Scientist – (Gen AI, Machine Learning): Are you energized by the idea of innovating with Generative AI? Do you want to create global impact while tackling challenges at the forefront of Artificial Intelligence? Do you dream of building ground-breaking products...Long term contractFull timeWorldwideShift work
- ...Number : R2868512 Position title : Lead Data Scientist Department: Commercial Data Science... ...transform Sanofi into a data-first and AI first organization by empowering everyone... ...advancements in AI, Generative AI, and Agentic AI, evaluating emerging technologies and identifying...Work at officeWork from homeHome officeFlexible hours
$80 - $120 per hour
...Mercor connects elite creative and technical talent with leading AI research labs. Headquartered in San Francisco, our investors... ...Position: Compliance / regulatory response with financial-services AI Evaluator Type: Contract Compensation: $80–$120/hour...Full timeContract workSummer workWork at officeRemote work$120 - $170 per hour
...creative and technical talent with leading AI research labs. Headquartered in San... ...Summers , and Jack Dorsey . Position: Data Science Expert Type: Contract... ...~ Prior experience with AI training , evaluation, or human-data projects. Application...Full timeContract workSummer workRemote work- ...Job role: As a data analyst, you will be responsible for compiling actionable insights from data and assisting program, sales and marketing managers build data-driven processes. Your role will involve driving initiatives to optimize for operational excellence and revenue...Full time
$168k - $231k per year
...At Faire, we're using the power of tech, data, and machine learning to connect this thriving... ...stores. Our highly skilled team of data scientists and machine learning engineers specialize... ...Improve Faire’s Net terms portfolio by evaluating creditworthiness of retailers on Faire’s...Full timeWork at officeLocal areaRemote work2 days per week- ...Reference Number: R2868507 Position title: Data Scientist Manager Department: Commercial Data... ...transform Sanofi into a data-first and AI first organization by empowering everyone... ..., machine learning, and industry trends, evaluating new technologies and identifying opportunities...Work at officeWork from homeHome officeFlexible hours
$73.38 - $88.06 per hour
...RQ00746-Data Analytical Specialist/Scientist Location: 661 University Ave, Suite 1701, Toronto, ON M5G 1M1... ...sustainable knowledge transfer. AI Disclaimer: Source Code may use artificial... ...have qualifications. The client will evaluate candidates based on both rate...Full timeContract workWork at office$85 per hour
...Mercor connects elite creative and technical talent with leading AI research labs. Headquartered in San Francisco, our investors... ...Responsibilities Use frontier AI coding agents to complete and evaluate complex infrastructure engineering tasks. Review model-...Full timeContract workSummer workRemote work$70 per hour
...creative and technical talent with leading AI research labs. Headquartered in San Francisco... ...Remote Role Responsibilities Evaluate AI-generated responses to strengthen reasoning... ...written feedback to improve training data quality and downstream performance. Develop...Remote jobContract workSummer work$80 - $120 per hour
...Mercor connects elite creative and technical talent with leading AI research labs. Headquartered in San Francisco, our investors... ...Summers , and Jack Dorsey . Position: Software / AI / IT / data Evaluator Type: Contract Compensation: $80–$120/hour...Remote jobContract workSummer workWork at office
Do you want to receive more vacancies?
Subscribe and receive similar vacancies to Data Scientist - AI Evaluation. Be the first to apply!
Related searches
- scientifique de données Toronto, ON
- senior data scientist Toronto, ON
- junior data scientist Toronto, ON
- data scientist visa sponsorship Toronto, ON
- environmental data scientist Toronto, ON
- public health data scientist Toronto, ON
- data scientist no experience Toronto, ON
- data scientist Toronto, ON
- sap master data Toronto, ON
- data analytics engineer Toronto, ON