DevOps Engineer - AI Model Evaluator
$85 par heureMercor
Toronto, ON
- Travail à distance
About the job
Mercor connects elite creative and technical talent with leading AI research labs. Headquartered in San Francisco, our investors include Benchmark , General Catalyst , Peter Thiel , Adam D'Angelo , Larry Summers , and Jack Dorsey .
Position: DevOps / SRE / Cloud Engineer (Coding Agent Experience)
Type: Contract
Compensation: $85/hour
Location: Remote
Role Responsibilities
- Use frontier AI coding agents to complete and evaluate complex infrastructure engineering tasks.
- Review model-generated implementations involving cloud platforms , Kubernetes , CI/CD systems , observability , and infrastructure automation .
- Identify bugs, edge cases, reliability issues, and failure modes in model outputs.
- Compare outputs from multiple frontier models to assess their strengths and weaknesses.
- Apply professional engineering judgment to realistic infrastructure engineering scenarios.
Qualifications
Must-Have
- 2+ years of professional DevOps , SRE , or Cloud Engineering experience.
- Experience with AWS , Azure , GCP , Kubernetes , Terraform , CI/CD pipelines , or observability tooling.
- Regular use of AI coding agents such as Cursor, Claude Code, Codex, Windsurf, Gemini CLI, or similar tools.
- Ability to evaluate model-generated infrastructure and reliability engineering solutions.
Preferred
- Experience supporting production-scale systems.
Compensation & Legal
- $400 per accepted task
- Compensation tied to accepted work.
Application Process (Takes 20–30 mins to complete)
- Upload resume
- AI interview based on your resume
- Submit form
Resources & Support
- For details about the interview process and platform information, please check:
- For any help or support, reach out to: Montrer un mail
PS: Our team reviews applications daily. Please complete your AI interview and application steps to be considered for this opportunity.
L'offre d'emploi a été publiée Il y a 2 mois
Des emplois similaires qui pourraient vous intéresserBasé sur l'offre DevOps Engineer - AI Model Evaluator à Toronto, ON
$85 par heure
...creative and technical talent with leading AI research labs. Headquartered in San... ..., and Jack Dorsey . Position: DevOps / SRE / Cloud Engineer (Coding Agent Experience) Type:... ...AI coding agents to complete and evaluate complex infrastructure engineering tasks...SuggéréTravail à distanceСontratTravail d'été- ...Outlier helps the world’s most innovative companies improve their AI models by providing human feedback. Are you a strong communicator with fluency in English who would like to lend your expertise to train AI models? About the opportunity: Outlier is looking for talented...SuggéréTaux horaireСontratFreelanceStageTravail à distanceHoraires flexibles
$80 par heure
...and technical talent with leading AI research labs. Headquartered in... ...Jack Dorsey . Position: Data Engineer (Coding Agent Experience)... ...AI coding agents to complete and evaluate complex data engineering tasks. Review model-generated implementations involving...SuggéréTravail à distanceСontratTravail d'été- ...Palona’s AI agents operate in real restaurant environments: noisy phone lines... ...more than selecting the newest model. It requires disciplined evaluation, high-quality data, modeling judgment... ...looking for an applied AI Modeling Engineer to improve the intelligence, accuracy...SuggéréContrat Longue DuréeTemps pleinTravail temporaireStageRecrutement immédiat
- ...looking for mission-driven talent to help accelerate our growth. The Opportunity SWTCH is looking for a highly motivated DevOps Engineer to join our growing tech team. Reporting to the Director of Engineering, you will design, build, and optimize our AWS container platform...SuggéréTemps plein
$92k - $118k par année
...building secure, automated, and scalable DevOps solutions. Join Capco to help transform enterprise... ...data and analytics through cloud-native engineering and continuous delivery. The Role We... ...At Capco, we use artificial intelligence (AI) tools to support and enhance several...Temps pleinStageRecrutement immédiat$80k - $130k par année
Senior DevOps Engineering Consultant - Azure & AI Position Description The Senior DevOps Engineering consultant... ...and Kubernetes based deployment models. .Partner with engineering,... ...peuvent être utilisés pour aider à l’évaluation des candidatures, sans jamais remplacer...$140k - $175k par année
...Job Responsibility: About the team We are looking for brilliant staff DevOps engineers who have a passion for consumer products and want to join an energetic team that is changing the way people think about travel. We ship code daily, A/B test extensively, and have a primary...Temps pleinTravail au bureauZone localeTravail à distance- ...AGENT DEVOPS PLATFORM ENGINEER ROLE DESCRIPTIONS Design and operate AI agents that automate tasks across the software development lifecycle. Build and maintain... ...identity governance, and least-privilege access models. • Embed security, compliance, vulnerability...Emploi permanent
- ...platforms such as Amazon EKS and Google Kubernetes Engine (GKE). ~ Provision and manage cloud... ...and tooling using Python to support DevOps processes. ~ Monitor system performance,... ...ZoomInfo (NASDAQ: GTM), the all-in-one AI GTM platform, enables sales, marketing, and...
$72.06k - $138.94k par année
...At Capgemini Engineering, the world leader in engineering services, we... ...seeking a highly skilled Senior DevOps Engineer to help build, operate... ...through automation and AI-assisted operations. Your role... ...Understanding of ML lifecycle management, model deployment, and production...Emploi permanentTemps pleinZone localeTravail à distance$157.4k - $179.6k par année
...Toronto, Ontario,Manager, Credit Modeling Strategy & AnalyticsAbout... ...who come together to build our models and policies which lead to our... ...Finance, Statistics, Science, Engineering)\n \n At least 1 year of experience... ...of artificial intelligence (AI) to enhance the candidate...Contrat Longue DuréeTravail au bureauTravail à domicileCapital One - CA
Toronto, ONil y a 27 jours- ...CoreFactor is searching for a DevOps Engineer on a permanent/full-time basis. This position is hybrid and will require the successful incumbent... ...four (4) times per week. We are seeking an experienced AI/ML DevOps Engineer to join our clients Data, AI & Analytics team...Emploi permanentTemps pleinTravail au bureau
$100 - $150 par heure
...technical talent with leading AI research labs. Headquartered in... ...data analyses , statistical modeling work , machine learning... ..., and technical reports. Evaluate AI-generated or human-created... ...detailed written justifications for evaluations and scores to maintain...Travail à distanceСontratTravail d'été- ...Job Responsibility: We are looking for "Senior AWS DevOps Engineer" in Canada. Full-time Permanent/T4 (hourly) - 100% Remote Responsibilities:- By utilizing Python, NodeJs, CI/CD, GitOps Methodologies, etc., create and extend Full Stack infrastructure automation for enterprise...Taux horaireEmploi permanentTemps pleinTravail à distance
- ...Director, Engineering, AI Job Description Are you passionate about transforming how legal professionals... ...to a mission-based operating model. Replace project-by-project handoffs with... ...retrieval and model integration through evaluation, deployment, and monitoring. Set the...Contrat Longue DuréeTemps pleinСontratTravail au bureauZone localeHoraires flexiblesTravail posté
$140.6k - $190.6k par année
...Job Description: Engineering Manager, Are you passionate about transforming... ...future of legal research with AI? Join our Global Westlaw... ...(RAG), and large language models (LLMs). You will also champion... ...review processes. Champion a DevOps-first mindset in which...Temps pleinStageTravail au bureauZone localeLe monde entierHoraires flexiblesTravail posté- ...Our team is looking for a DevOps engineer with at least 6 years cloud and Kubernetes experience including specialization in ArgoCD. Our project involves building automations for a greenfield managed service product in AWS and we are looking for candidates who are interested...Temps plein
$109k - $173k par année
...City: Toronto Project Objectives & Role Summary The AI DevOps Specialist is primarily responsible for the technical... ...Celestica’s global HPS Artificial Intelligence (AI) sandbox, modeling, and software engineering toolchains. This specialist will bridge the gap...Travail temporaireZone localeTravail à distance- ...DevOps Engineer About us TitanFile's mission is to democratize secure client communication. This is a challenge worthy of pursuit! The costs... ...problems. We expect our engineers to work fluently with AI tooling and to apply the same engineering judgment to what it produces...Temps pleinTravail au bureauHoraires flexibles
- ...Number : R2868511 Position title : AI Engineer Department: Commercial Data Science... ...enterprise data foundation strategies Monitor, evaluate, and continuously improve AI application... ...emerging AI technologies, foundational models, frameworks, and engineering practices,...Travail au bureauTravail à domicileBureau à domicileHoraires flexibles
- ...staging, and production. Working with developers, architects, testers, cybersecurity specialists, and operational teams, the Senior DevOps Engineer improves deployment consistency, reduces manual effort, strengthens traceability, and supports secure and repeatable releases....Contrat Longue DuréeСontratStage
$95 par heure
...Experience 5 years or more Hybrid Work must be completed both in person and remotely. Experience and specialization Computer and technology knowledge Cloud Development and operations (DevOps) Jira Amazon Web Services (AWS) Atlassian Confluence API...Emploi permanentTemps pleinTravail à distance- ...We are seeking a Lead Azure DevOps Engineer with deep expertise in Azure Networking to drive the design, implementation, and optimization of... ...conform to best practices for Terraform IaC creation; a generic AI-developed Terraform solution is not enough. Ability to set the right...Travail à distance
$100 par heure
...Mindrift connects specialists with project-based AI opportunities for leading tech companies, focused on testing, evaluating, and improving AI systems. Participation is... ...evaluate clinical EHR vignettes for a medical AI evaluation program. General and internal medicine are our...Taux horaireEmploi permanentTemps partiel10 heures semaine- ...Our team is looking for a DevOps engineer with at least 7 years cloud experience including specialization with Identity Our project involves building automations for a greenfield managed service product in AWS and we are looking for candidates who are interested in designing...Temps plein
- ...Senior AI Engineer Toronto Downtown, ON - Hybrid (4 Days WFO) In Person Interview Required... ...LLM pipelines, Agentic workflows, Model Context Protocol (MCP), Graph/RAG architectures... ...using Apache Airflow/DAGs. • Cloud & DevOps (AWS): Comprehensive knowledge of AWS infrastructure...Contrat Longue DuréeEmploi permanent
- ...Cloud Platform Engineer – AWS, Python, DevOps & AI/LLM Required Skillsets: • 3–5+ years of hands-on CI/CD, cloud, and platform engineering... ...: prompt management, tool use, agent architectures, evaluation, observability, and reliability. • Experience with Git...Emploi permanent
$90k - $115k par année
...greener and smarter mobility worldwide, connecting cities as we reduce carbon and replace cars. Could you be the full-time DevOps Engineer in Toronto, ON we’re looking for? Your future role We’re looking for a motivated DevOps Engineer to help modernize and...Contrat Longue DuréeTemps pleinStageLe monde entier$84k - $126k par année
...Job Type: Permanent Work Model: Hybrid Reference code: 133157 Primary Location: Toronto, ON All Available Locations: Toronto... ...and hiring teams will utilize human screening combined with AI technology to help identify the skills and qualities that matter most...Emploi permanentHoraires flexibles
Voulez-vous recevoir plus d'offres d'emploi ?
S'abonner et recevoir des offres d'emploi similaires à DevOps Engineer - AI Model Evaluator. Soyez parmi les premiers à postuler !
Recherches connexes