Data Scientist AI Evaluation
$100 - $150 per hourFull-time
Mercor
About the job
Mercor connects elite creative and technical talent with leading AI research labs. Headquartered in San Francisco our investors include Benchmark General Catalyst Peter Thiel Adam DAngelo Larry Summers and Jack Dorsey .
Position: Data Scientist Talent Network
Type: Contract
Compensation: $100$150/hour
Location: Remote
Role Responsibilities
- Design precise task-specific grading criteria for data science deliverables including exploratory data analyses statistical modeling work machine learning pipelines and A/B test write-ups .
- Evaluate AI-generated or human-created work against established criteria to ensure quality and consistency.
- Provide detailed written justifications for evaluations and scores to maintain transparency and accountability.
- Apply consistent evidence-based judgment to ensure assessments are reproducible and defensible.
- Incorporate structured feedback from senior reviewers and iterate on submitted work for continuous improvement.
- Work independently and asynchronously to meet deadlines and improve AI model performance .
Qualifications
Must-Have
- 1 years of professional data science experience.
- Experience at a leading technology research or quantitative firm.
- Strong command of Python SQL statistical modeling machine learning and experimentation .
- Exceptional written communication skills.
- Detail-oriented and consistent approach to evaluating complex work.
- Comfort receiving feedback and calibrating judgment against established standards.
Application Process (Takes 2030 mins to complete)
- Upload resume
- AI interview based on your resume
- Submit form
Resources & Support
- For details about the interview process and platform information please check:
- For any help or support reach out to:
PS: Our team reviews applications daily. Please complete your AI interview and application steps to be considered for this opportunity.
Vacancy posted 23 hours ago
Similar jobs that could be interesting for youBased on the Data Scientist AI Evaluation in Toronto, ON vacancy
$100 - $150 per hour
...creative and technical talent with leading AI research labs. Headquartered in San... ...Summers , and Jack Dorsey . Position: Data Scientist Talent Network Type: Contract... ...pipelines , and A/B test write-ups . Evaluate AI-generated or human-created work...SuggestedRemote jobContract workSummer work$80 per hour
...creative and technical talent with leading AI research labs. Headquartered in San Francisco... ..., and Jack Dorsey . Position: Data Engineer (Coding Agent Experience) Type... ...frontier AI coding agents to complete and evaluate complex data engineering tasks. Review...SuggestedRemote jobContract workSummer work$120 - $170 per hour
...creative and technical talent with leading AI research labs. Headquartered in San... ...Summers , and Jack Dorsey . Position: Data Science Expert Type: Contract... ...~ Prior experience with AI training , evaluation, or human-data projects. Application...SuggestedRemote jobContract workSummer work- ...Tiger Analytics is looking for experienced Data Scientists GenAI to join our fast-growing advanced... ...in Data Science, Machine Learning and AI. We are the trusted analytics partner for... ..., JumpStart, HuggingFace ~ Experience evaluating ethical implications of AI and...SuggestedFull timeLocal area
$120k - $160k per year
...trusted platforms for financial data access, enrichment, and money... ...We're hiring a Senior Data Scientist to own machine-learning models... ...retraining, champion-challenger evaluation and promotion. Partner... ...and rollout. Move fast with AI-assisted development -we use...SuggestedContract workApprenticeshipRemote workFlexible hours$80 - $120 per hour
...Mercor connects elite creative and technical talent with leading AI research labs. Headquartered in San Francisco, our investors... ...Summers , and Jack Dorsey . Position: Software / AI / IT / data Evaluator Type: Contract Compensation: $80–$120/hour...Remote jobContract workSummer workWork at office$80k - $180k per year
...Zafin is an AI platform company helping regulated institutions modernize how critical... ..., an integration platform that connects data, systems, and workflows across complex enterprise... .... What’s the Opportunity? The AI Evaluation Engineer ensures AI agent solutions are accurate...Full time$39 per hour
...creative and technical talent with leading AI research labs. Headquartered in San Francisco... .../week Role Responsibilities Evaluate AI model output lyrics , voice generation... ...Score the quality of musical training data to enhance model performance. Write music...Remote jobContract workSummer workImmediate start$80 - $120 per hour
...Mercor connects elite creative and technical talent with leading AI research labs. Headquartered in San Francisco, our investors... ...Position: Compliance / regulatory response with financial-services AI Evaluator Type: Contract Compensation: $80–$120/hour...Remote jobContract workSummer workWork at office- ...of your career. About the team Our Data Science team partners deeply with teams across... ...grow responsibly. We're looking for data scientists with a passion for analyzing data,... ...fill it up anywhere, and even an agentic AI-wallet that allows your preferred AI model...Local area
- ...of your career. About the team Our Data Science team partners deeply with teams across... ...grow responsibly. We're looking for data scientists with a passion for analyzing data,... ...actionable recommendations Proficiency with AI tools to accelerate model development, analysis...
$140k - $175k per year
...HelloFresh to drive smarter marketing through AI-powered automation, intelligent systems,... ...S'more about the team As a Senior Data Scientist in AdTech, you will design, build, and deploy... ...technology to help us process and evaluate applications more efficiently. This includes...Work at officeRemote workWork from homeFlexible hours2 days per week3 days per week$69.7k - $98.4k per year
...) team partners with business leaders to transform data into actionable insights, analytics solutions, and AI-driven capabilities that support strategic decision... ...and deliver measurable business value. The Data Scientist I will support the development and delivery of analytics...Full timeWork from home$70 - $90 per hour
...connects elite creative and technical talent with leading AI research labs. Headquartered in San Francisco, our... ...large language models . Create high-quality data by developing difficult problems in your domain. Evaluate and improve AI model performance through rigorous...Remote jobContract workSummer work$96k - $125k per year
...focused problem solvers who thrive on complex data challenges. Guided by our values of Care,... ...-5 years of hands-on experience as a Data Scientist ~ Proficiency in Python and SQL, with... ...with LLMs, prompt engineering, and AI application development using APIs (e.g. OpenAI...Full time- ...Tiger Analytics is pioneering what AI and analytics can do to solve some of the toughest... .... We develop bespoke solutions powered by data and technology for several Fortune 100... ...space. We are looking for a Senior Data Scientist with a good blend of data analytics background...Full timeLocal areaRemote work
$85 per hour
...Mercor connects elite creative and technical talent with leading AI research labs. Headquartered in San Francisco, our investors... ...Responsibilities Use frontier AI coding agents to complete and evaluate complex infrastructure engineering tasks. Review model-...Remote jobContract workSummer work- ...Reference no. R2862231 Position title: Data and AI Engineer Department: Data... ...to making key datasets available to data scientists, analysts, and consumers throughout the... ..., and activities related to performance evaluation. Information collected and processed by...Full timeWork from homeFlexible hours
- ...Founded in 2013, Fusemachines is a global provider of enterprise AI products and services, on a mission to democratize AI.... ...achieve their full potential with AI. About the role A Lead Data Scientist is responsible for designing and implementing data-driven solutions...Full timeLocal area
$72k - $138k per year
...analytical and client-focused Senior Consultant to join our team as a Data Scientist / Power Systems Analyst. This role is ideal for a professional... ...• Stay informed on trends in power systems, utility analytics, AI/ML, energy markets, and grid modernization. • Support proposal...Permanent employmentFlexible hours$110k - $160k per year
...a better working world for all. As AI Engineer, you will design and build production... ...workflows, multi-agent coordination, evaluation frameworks, observability and governance... ...Architect, AI Platform Engineers, and Data Scientists to translate complex business challenges...Long term contract$72k - $138k per year
...will your typical day look like? As a Senior Consultant on the AI & Data Transformation team, you will operate at the intersection of... ...recommendations. • Collaborating with cross-functional teams of data scientists, engineers, designers, and consultants to deliver scalable AI...Permanent employmentFlexible hours$73.38 - $88.06 per hour
...RQ00746-Data Analytical Specialist/Scientist Location: 661 University Ave, Suite 1701, Toronto, ON M5G 1M1... ...sustainable knowledge transfer. AI Disclaimer: Source Code may use artificial... ...have qualifications. The client will evaluate candidates based on both rate...Contract workWork at office$108k - $135k per year
...We aim to achieve this by cultivating a work environment where all team members belong and have the opportunity to thrive. As a Data Scientist in the Pay, Integrity & Identity org, you will collaborate with our world class team of engineers, product managers, analysts and...Hourly payWork at officeFlexible hours3 days per week$70 per hour
Role: Data Scientist Rates: Up To $70.00 p/h INC. Structure: 12 month contract Location: North York, 1-2 days per month on-site --... ...Experience building, training, and deploying models using Vertex AI, BigQuery, and Google Cloud Storage. • Communication & Critical...Contract work$180k - $220k per year
...and beyond. Join our global Data Science team and take a leadership... ...marketplace through cutting-edge AI and machine learning. As a Principal Data Scientist, you’ll spearhead high-impact AI initiatives... ..., deep learning), including model evaluation, explainability, and selection for...Long term contractWorldwide- ...Principal Data Scientist – (Gen AI, Machine Learning): Are you energized by the idea of innovating with Generative AI? Do you want to create global impact while tackling challenges at the forefront of Artificial Intelligence? Do you dream of building ground-breaking products...Full timeWorldwideShift work
$68k - $72k per year
...fill a new position of Manager, Planning and Evaluation. This is a full-time position working 35... ..., including workflow development, data tracking, analysis, and reporting. ~... ...As part of our recruitment process, we use AI-enabled tools to assist with summarizing applicant...Full timeInternshipWork at officeFlexible hoursWeekend work$216k - $297k per year
...Secure Every Identity, from AI to Human Identity is the key... ...identity and security. We believe data, and developing our own models... ...for a Principal Applied AI Scientist to take product ideas from 0 to... ...models Create and maintain evaluation and training datasets and...Work at officeLocal areaWorldwide$55 - $80 per hour
...creative and technical talent with leading AI research labs. Headquartered in San Francisco... ...models . Create high-quality data to inform the future of AI innovation by... ...challenging problems in your domain. Evaluate and improve AI model training and outputs...Remote jobContract workSummer work
Do you want to receive more vacancies?
Subscribe and receive similar vacancies to Data Scientist AI Evaluation. Be the first to apply!
Related searches
- scientifique de données Toronto, ON
- environmental data scientist Toronto, ON
- senior data scientist Toronto, ON
- junior data scientist Toronto, ON
- public health data scientist Toronto, ON
- data scientist no experience Toronto, ON
- data scientist Toronto, ON
- data analytics Toronto, ON
- data Toronto, ON
- clinical data coordinator remote Toronto, ON