Data Engineer - AI Model Evaluator
$80 per hourMercor
Toronto, ON
- Remote job
About the job
Mercor connects elite creative and technical talent with leading AI research labs. Headquartered in San Francisco, our investors include Benchmark , General Catalyst , Peter Thiel , Adam D'Angelo , Larry Summers , and Jack Dorsey .
Position: Data Engineer (Coding Agent Experience)
Type: Contract
Compensation: $80/hour
Location: Remote
Role Responsibilities
- Use frontier AI coding agents to complete and evaluate complex data engineering tasks.
- Review model-generated implementations involving ETL pipelines , data warehouses , analytics platforms , and distributed data systems .
- Identify bugs, edge cases, scalability issues, and failure modes in model outputs.
- Compare outputs from multiple frontier models and assess their strengths and weaknesses.
- Apply professional engineering judgment to realistic data engineering scenarios.
Qualifications
Must-Have
- 2+ years of professional data engineering experience.
- Experience building ETL pipelines , data warehouses , analytics platforms , or distributed data systems .
- Regular use of AI coding agents such as Cursor, Claude Code, Codex, Windsurf, Gemini CLI, or similar tools.
- Ability to evaluate model-generated data infrastructure and pipeline implementations.
Preferred
- Experience operating large-scale data platforms .
Compensation & Legal
- $400 per accepted task .
- Compensation is tied to accepted work.
Application Process (Takes 20–30 mins to complete)
- Upload resume
- AI interview based on your resume
- Submit form
Resources & Support
- For details about the interview process and platform information, please check:
- For any help or support, reach out to: Show email
PS: Our team reviews applications daily. Please complete your AI interview and application steps to be considered for this opportunity.
Vacancy posted 6 days ago
Similar jobs that could be interesting for youBased on the Data Engineer - AI Model Evaluator in Toronto, ON vacancy
$80 per hour
...creative and technical talent with leading AI research labs. Headquartered in San... ...Summers and Jack Dorsey . Position: Data Engineer (Coding Agent Experience) Type:... ...frontier AI coding agents to complete and evaluate complex data engineering tasks. Review...SuggestedFull timeContract workSummer workRemote work$85 per hour
...and technical talent with leading AI research labs. Headquartered in... ...Position: DevOps / SRE / Cloud Engineer (Coding Agent Experience)... ...coding agents to complete and evaluate complex infrastructure engineering tasks. Review model-generated implementations involving...SuggestedRemote jobContract workSummer work- ...Palona’s AI agents operate in real restaurant environments: noisy phone lines, varied accents, complex menus... ...systems requires more than selecting the newest model. It requires disciplined evaluation, high-quality data, modeling judgment, experimentation, and production...SuggestedLong term contractFull timeTemporary workInternshipImmediate start
$100 - $150 per hour
...technical talent with leading AI research labs. Headquartered in... ...Jack Dorsey . Position: Data Scientist Talent Network... ...data analyses , statistical modeling work , machine learning pipelines... ..., and technical reports. Evaluate AI-generated or human-created...SuggestedRemote jobContract workSummer work$100 - $150 per hour
...technical talent with leading AI research labs. Headquartered in... ...Jack Dorsey . Position: Data Scientist Talent Network... ...exploratory data analyses statistical modeling work machine learning... ...and A/B test write-ups . Evaluate AI-generated or human-created...SuggestedFull timeContract workSummer workRemote work- ...Senior Software Engineer Job Type: Contractor (~15 hours/week) Location: Remote... ...Senior Software Engineers to support an AI training project by creating reinforcement learning environments that evaluate AI models on complex software engineering tasks using...Remote jobFor contractors
$120 - $170 per hour
...technical talent with leading AI research labs. Headquartered in... ...Jack Dorsey . Position: Data Science Expert Type:... ...deliverables such as analyses, models, dashboards, and experiment readouts... ...experience with AI training , evaluation, or human-data projects....Remote jobContract workSummer work$80k - $180k per year
...Zafin is an AI platform company helping regulated institutions modernize how critical... ..., an integration platform that connects data, systems, and workflows across complex enterprise... .... What’s the Opportunity? The AI Evaluation Engineer ensures AI agent solutions are accurate,...Full time- ...Reference no. R2867104 Position title: Lead Data and AI Engineer Department: US Commercial Data Engineering Location: Toronto, ON... ...Informatica/IICS experience preferred Advanced SQL and data modeling skills, with working knowledge of Python and scripting...Long term contractWork from home
- ...inclusive world. Job Description The IT Data Modeler is responsible for designing, developing,... ...Collaborate with Business, Architecture, Engineering, and Analytics teams. Required... ...fueled by its market leading capabilities in AI, generative AI, cloud and data, combined...Permanent employmentFull timeLocal area
$95k - $145k per year
Data Modeler Position Description This role is hybrid and requires you to be at a Client... ...modelling and data product design and 25% data engineering enablement. Your future duties and... .... To support our recruitment team, AI tools may be used to help assess applications...Work at office$39 per hour
...creative and technical talent with leading AI research labs. Headquartered in San... ...hours/week Role Responsibilities Evaluate AI model output lyrics , voice generation, and other... ...Score the quality of musical training data to enhance model performance. Write...Remote jobContract workSummer workImmediate start$101k - $169k per year
...Job Type: Permanent Work Model: Hybrid Reference code: 131871 Primary Location... ...complex, high-impact engagements in the AI Model Risk Space, ensuring alignment with... ...Generative AI offerings, including testing, evaluating and benchmarking different LLMs and toolchains...Permanent employmentContract workFlexible hours$80 - $120 per hour
...creative and technical talent with leading AI research labs. Headquartered in San... ...and Jack Dorsey . Position: Data quality / CRM operations Evaluator Type: Contract... ...structured written feedback to improve AI model outputs . Apply deep subject-matter...Full timeContract workSummer workWork at officeRemote work$70 per hour
...technical talent with leading AI research labs. Headquartered in... ...Role Responsibilities Evaluate AI-generated responses to strengthen... ...reasoning and rigor in model outputs . Provide structured... ...feedback to improve training data quality and downstream performance...Remote jobContract workSummer work$80 - $120 per hour
...creative and technical talent with leading AI research labs. Headquartered in San... ...and Jack Dorsey . Position: Data quality / CRM operations Evaluator Type: Contract... ...structured written feedback to improve AI model outputs . Apply deep subject-matter...Remote jobContract workSummer workWork at office$110k - $160k per year
...a better working world for all. As AI Engineer, you will design and build production-grade... ...workflows, multi-agent coordination, evaluation frameworks, observability and governance... ...AI Architect, AI Platform Engineers, and Data Scientists to translate complex business...Long term contract$70 per hour
...technical talent with leading AI research labs. Headquartered in... ...Role Responsibilities Evaluate AI-generated responses to strengthen... ...reasoning and rigor in model outputs . Provide structured... ...feedback to improve training data quality and downstream performance...Remote jobContract workSummer work$120 - $170 per hour
...technical talent with leading AI research labs. Headquartered in... ...Jack Dorsey . Position: Data Science Expert Type:... ...deliverables, including analyses, models, dashboards, and experiment readouts... ...with AI training , evaluation, or human-data projects....Remote jobContract workSummer work$70 per hour
...creative and technical talent with leading AI research labs. Headquartered in San... ...answers. Calibrate tasks against frontier models, ensuring tasks ship only when strong models... ...Prior scientific software or research engineering experience. Start Date ~ Immediate...Remote jobContract workSummer workImmediate start$80 - $120 per hour
...Mercor connects elite creative and technical talent with leading AI research labs. Headquartered in San Francisco, our investors... ...Summers , and Jack Dorsey . Position: Software / AI / IT / data Evaluator Type: Contract Compensation: $80–$120/hour...Remote jobContract workSummer workWork at office$56k - $84k per year
...Job Type: Permanent Work Model: Hybrid Reference code: 133514 Primary Location... ...someone with: Relevant experience in AI/ML domain Knowledge of NLP, LLMs (extractive... ...AI offerings, including testing, evaluating and benchmarking different LLMs and toolchains...Permanent employmentContract workFlexible hours$85k - $156k per year
...Job Type: Permanent Work Model: Remote Reference code: 134003... ...Deloitte Global is the engine of the Deloitte network. Our professionals... ...We are seeking a Data Domain Modeler to define, standardize... ...human screening combined with AI technology to help identify the...Permanent employmentRemote workFlexible hours$80 - $120 per hour
...creative and technical talent with leading AI research labs. Headquartered in San... ...regulatory response with financial-services AI Evaluator Type: Contract Compensation... ...structured written feedback to improve AI model outputs. Apply deep subject-matter...Remote jobContract workSummer workWork at office- ...· Python · PySpark · SQL · Data Modeling Techniques · ETL/ELT Development... ...data modeling, advanced analytics, data engineering, and product delivery to transform complex... ...platforms to support analytics, reporting, and AI-driven business solutions. · This role...Contract work
- ...commercial insurance organization, is seeking an experienced Senior Data Modeler to join their enterprise data team. In this role, you will be... .... You will collaborate closely with Data Architects, Data Engineers, BI teams, and key business stakeholders across domains such as...Immediate start
$100 - $150 per hour
...technical talent with leading AI research labs. Headquartered in... .... Position: Sales Engineering Expert Type: Contract... ...supporting proofs of concept, and evaluating product fit and technical risk... ...infrastructure, cybersecurity, data platforms, developer tools, or...Remote jobContract workSummer workusd84.46k - usd100k per year
...Senior Data Engineer Join Livingston and grow your career in the constantly changing world... ...sources. Architect and optimize data models and data warehouse/lakehouse structures to... ...reporting, forecasting, advanced analytics and AI. Build and maintain the infrastructure...Long term contractFull timeFlexible hours$135k - $210k per year
...Overview: Guidepoint seeks an experienced Data/AI Engineer as an integral member of the Toronto-based AI team. The Toronto Technology Hub... ...scale Generative AI-powered systems, including large language model (LLM) applications and research agents, while ensuring the integration...Full timeInternship$72k - $138k per year
...Job Type: Permanent Work Model: Hybrid Reference code: 130069 Primary... ...-- We are looking for a passionate AI Research Engineer to join our team. You will work at the... ...research and product engineering—designing, evaluating, and deploying generative AI (GenAI) systems...Permanent employmentFlexible hours
Do you want to receive more vacancies?
Subscribe and receive similar vacancies to Data Engineer - AI Model Evaluator. Be the first to apply!
Related searches
- remote data engineer Toronto, ON
- python data engineer Toronto, ON
- data integration developer Toronto, ON
- gcp data engineer Toronto, ON
- data centre engineer Toronto, ON
- information engineer Toronto, ON
- data engineer Toronto, ON
- data cabling engineer Toronto, ON
- data engineering intern summer Toronto, ON
- junior data engineer Toronto, ON