Data Scientist - AI Evaluation
$100 - $150 per hourFull-time
Mercor
About the job
Mercor connects elite creative and technical talent with leading AI research labs. Headquartered in San Francisco, our investors include Benchmark , General Catalyst , Peter Thiel , Adam D'Angelo , Larry Summers , and Jack Dorsey .
Position: Data Scientist Talent Network
Type: Contract
Compensation: $100–$150/hour
Location: Remote
Role Responsibilities
- Design precise, task-specific grading criteria for data science deliverables, including exploratory data analyses , statistical modeling work , machine learning pipelines , and A/B test write-ups .
- Evaluate AI-generated or human-created work against established criteria to ensure quality and consistency.
- Provide detailed written justifications for evaluations and scores to maintain transparency and accountability.
- Apply consistent, evidence-based judgment to ensure assessments are reproducible and defensible.
- Incorporate structured feedback from senior reviewers and iterate on submitted work for continuous improvement.
- Work independently and asynchronously to meet deadlines and improve AI model performance .
Qualifications
Must-Have
- 1+ years of professional data science experience.
- Experience at a leading technology, research, or quantitative firm.
- Strong command of Python , SQL , statistical modeling , machine learning , and experimentation .
- Exceptional written communication skills.
- Detail-oriented and consistent approach to evaluating complex work.
- Comfort receiving feedback and calibrating judgment against established standards.
Application Process (Takes 20–30 mins to complete)
- Upload resume
- AI interview based on your resume
- Submit form
Resources & Support
- For details about the interview process and platform information, please check:
- For any help or support, reach out to: View email address on jobs.jobcopilot.com
PS: Our team reviews applications daily. Please complete your AI interview and application steps to be considered for this opportunity.
Vacancy posted a month ago
Similar jobs that could be interesting for youBased on the Data Scientist - AI Evaluation in Toronto, ON vacancy
$100 - $150 per hour
...creative and technical talent with leading AI research labs. Headquartered in San... ...Summers , and Jack Dorsey . Position: Data Scientist Talent Network Type: Contract... ...pipelines , and technical reports. Evaluate AI-generated or human-created work against...SuggestedRemote jobContract workSummer work$120 - $170 per hour
...creative and technical talent with leading AI research labs. Headquartered in San... ...Summers , and Jack Dorsey . Position: Data Science Expert Type: Contract... ...~ Prior experience with AI training , evaluation, or human-data projects. Application...SuggestedFull timeContract workSummer workRemote work$80 per hour
...creative and technical talent with leading AI research labs. Headquartered in San Francisco... ..., and Jack Dorsey . Position: Data Engineer (Coding Agent Experience) Type... ...frontier AI coding agents to complete and evaluate complex data engineering tasks. Review...SuggestedFull timeContract workSummer workRemote work$96k - $125k per year
...focused problem solvers who thrive on complex data challenges. Guided by our values of Care,... ...-5 years of hands-on experience as a Data Scientist ~ Proficiency in Python and SQL, with... ...with LLMs, prompt engineering, and AI application development using APIs (e.g. OpenAI...SuggestedFull time- ...Position: Data Scientist Location: Toronto, ON Job ID#: RQ11452 Duration: 12 Months Role Overview We are seeking... ...them staff difficult assignments with proven Experts uniquely evaluated by our proven QMS- Quality Management System Staff Development...SuggestedFull time
$100k - $140k per year
...The role We're looking for a driven and entrepreneurial Data Scientist to join our Data team. You will own the full machine learning lifecycle... ...only and does not include bonus, or other benefits. We use AI in our hiring process. Learn more about us at #tripstack #LI...Full timeWork at officeFlexible hours$68.8k - $100k per year
...12-month contract opportunity* The Data Scientist will be a pivotal member of the Enterprise... ...and implementing advanced analytics and AI models. Leverage statistical techniques,... ...Critical Thinking – The ability to analyze, evaluate and interpret information from various...Full timeContract workWork at officeRemote work$245 - $280 per day
...Canada. Pay: $245$280/hr Role Title: Data Scientist Role Type: Contractor Location:... ...the company is partnering with a leading AI lab to hire data scientists and quantitative... ...reasoning. Scope of Work # Evaluate AI model outputs on data science statistics...Full timeFor contractorsRemote work- ...of your career. About the team Our Data Science team partners deeply with teams across... ...grow responsibly. We're looking for data scientists with a passion for analyzing data,... ...actionable recommendations Proficiency with AI tools to accelerate model development, analysis...
$80 - $100 per hour
...Mercor connects elite creative and technical talent with leading AI research labs. Headquartered in San Francisco, our investors include... ...D'Angelo , Larry Summers , and Jack Dorsey . Position: Data Science Documents Type: Contract Compensation:...Full timeContract workSummer workRemote work$80.7k - $100k per year
..., enforcement, and education. We help build a safer community. Data analytics sits at the heart of TSSA’s vision for a safer Ontario.... ...with us. Your Key Contributions: Identify opportunities for AI and advanced analytics , applying data science techniques to...Full time- ...strong, globally distributed team from the start. We need a Data Scientist to help design and build the algorithms that power insights, pre... ...trends, and opportunities for product value. Continuously evaluate model performance and iterate to improve accuracy, reliability...Full timePart time
$120k - $160k per year
...trusted platforms for financial data access, enrichment, and money... ...We're hiring a Senior Data Scientist to own machine-learning models... ...retraining, champion-challenger evaluation and promotion. Partner... ...and rollout. Move fast with AI-assisted development -we use...Full timeContract workApprenticeshipRemote workFlexible hours- ...Zafin is an AI platform company helping regulated institutions modernize how critical... ..., an integration platform that connects data, systems, and workflows across complex enterprise... .... What’s the Opportunity? The AI Evaluation Engineer ensures AI agent solutions are accurate...Full time
$180k - $220k per year
...and beyond. Join our global Data Science team and take a leadership... ...marketplace through cutting-edge AI and machine learning. As a Principal Data Scientist, you’ll spearhead high-impact AI initiatives... ..., deep learning), including model evaluation, explainability, and selection for...Long term contractWorldwide- ...Company Description You will join a world-class team of engineers and data scientists from Facebook, Uber, Amazon and Google. We are a fast growing consulting firm based in Toronto with clients ranging from leading startups building impactful technologies to Fortune 500...Full time
- ...successful digital experiences and client partnerships. As a QA / AI Evaluation Engineer, you will join a highly motivated and experienced team... .... ~4+ years in QA / test engineering, with exposure to data/ML systems. ~ Strong Python and data-science techniques for measuring...Full timeImmediate start
- ...Principal Data Scientist – (Gen AI, Machine Learning): Are you energized by the idea of innovating with Generative AI? Do you want to create global impact while tackling challenges at the forefront of Artificial Intelligence? Do you dream of building ground-breaking products...Long term contractFull timeWorldwideShift work
$80 - $120 per hour
...connects elite creative and technical talent with leading AI research labs. Headquartered in San Francisco, our investors... ..., Larry Summers , and Jack Dorsey . Position: Data quality / CRM operations Evaluator Type: Contract Compensation: $80–$120/hour...Full timeContract workSummer workWork at officeRemote work- ...we’re shaping the future of automotive retail in Canada and beyond. As Senior Data Scientist , you will be key contributor within our global Data Science organization, shaping AI and machine learning systems that sit at the core of our marketplace platforms across...Long term contractPermanent employmentFull time
$60 - $100 per hour
...creative and technical talent with leading AI research labs. Headquartered in San... ...Jack Dorsey . Position: Exceptional Data Scientists Type: Contract Compensation... ...to build predictive models. Evaluate and improve AI models using TensorFlow...Full timeContract workSummer workRemote work$100k per year
...About the Job: \ We are seeking a business -focused Data Scientist \ to use data science, advanced analytics, Machine Learning, and AI to generate insights, improve customer experience, optimize operations, and support strategic decision -making across business...Permanent employmentFull timeWork at officeLocal area- ...of your career. About the team Our Data Science team partners deeply with teams across... ...grow responsibly. We're looking for data scientists with a passion for analyzing data,... ...actionable recommendations Proficiency with AI tools to accelerate model development, analysis...
$120k - $150k per year
...DataVisor DataVisor is the world's leading AI-powered Fraud and Risk Platform that... ...easy consolidation and enrichment of any data, DataVisor's fraud and anti-money laundering... ...We are looking for a Senior-Level Data Scientist to join our Fraud Detection team — someone...Full time$100 - $150 per hour
...creative and technical talent with leading AI research labs. Headquartered in San Francisco... ...and Jack Dorsey . Position: Data Science Expert Type: Contract... ...Remote Role Responsibilities Evaluate AI-generated artifacts for usability in...Full timeContract workSummer workWork at officeRemote work$80 - $120 per hour
...Mercor connects elite creative and technical talent with leading AI research labs. Headquartered in San Francisco, our investors... ...Position: Compliance / regulatory response with financial-services AI Evaluator Type: Contract Compensation: $80–$120/hour...Full timeContract workSummer workWork at officeRemote work$168.3k - $217.8k per year
...your home — from personalized guidance to AI tools and a best-in-class hiring experience... ...strategy and team execution, and we use data to tell Thumbtack’s story to our team, our... ...impact. About the role As a Sr. Data Scientist on Business Analytics, you’ll drive...Full timeSeasonal workLocal area$80 - $120 per hour
...Mercor connects elite creative and technical talent with leading AI research labs. Headquartered in San Francisco, our investors... ...Position: Compliance / regulatory response with financial-services AI Evaluator Type: Contract Compensation: $80–$120/hour...Full timeContract workSummer workWork at officeRemote work- ...Job role: As a data analyst, you will be responsible for compiling actionable insights from data and assisting program, sales and marketing managers build data-driven processes. Your role will involve driving initiatives to optimize for operational excellence and revenue...Full time
$20 - $160 per hour
...Mercor connects elite creative and technical talent with leading AI research labs. Headquartered in San Francisco, our investors... ...Position: Generalist Annotator — Health AI Conversation Quality Evaluation Type: Contract Compensation: $20–$160/hour...Remote jobContract workSummer work
Do you want to receive more vacancies?
Subscribe and receive similar vacancies to Data Scientist - AI Evaluation. Be the first to apply!
Related searches
- scientifique de données Toronto, ON
- data scientist visa sponsorship Toronto, ON
- junior data scientist Toronto, ON
- data scientist no experience Toronto, ON
- public health data scientist Toronto, ON
- data scientist Toronto, ON
- environmental data scientist Toronto, ON
- senior data scientist Toronto, ON
- data Toronto, ON
- salesforce data migration Toronto, ON