DevOps Engineer - AI Model Evaluator
$85 par heureMercor
Toronto, ON
- Travail à distance
About the job
Mercor connects elite creative and technical talent with leading AI research labs. Headquartered in San Francisco, our investors include Benchmark , General Catalyst , Peter Thiel , Adam D'Angelo , Larry Summers , and Jack Dorsey .
Position: DevOps / SRE / Cloud Engineer (Coding Agent Experience)
Type: Contract
Compensation: $85/hour
Location: Remote
Role Responsibilities
- Use frontier AI coding agents to complete and evaluate complex infrastructure engineering tasks.
- Review model-generated implementations involving cloud platforms , Kubernetes , CI/CD systems , observability , and infrastructure automation .
- Identify bugs, edge cases, reliability issues, and failure modes.
- Compare outputs from multiple frontier models and assess their strengths and weaknesses.
- Apply professional engineering judgment to realistic infrastructure engineering scenarios.
- Work independently and asynchronously to meet sprint-based project deadlines.
Qualifications
Must-Have
- 2+ years of professional DevOps , SRE , or Cloud Engineering experience.
- Experience with AWS , Azure , GCP , Kubernetes , Terraform , CI/CD pipelines , or observability tooling .
- Regular use of AI coding agents such as Cursor, Claude Code, Codex, Windsurf, Gemini CLI, or similar tools.
- Ability to evaluate model-generated infrastructure and reliability engineering solutions.
Preferred
- Experience supporting production-scale systems.
Compensation & Legal
- $400 per accepted task .
- Compensation is tied to accepted work.
Application Process (Takes 20–30 mins to complete)
- Upload resume
- AI interview based on your resume
- Submit form
Resources & Support
- For details about the interview process and platform information, please check:
- For any help or support, reach out to: Montrer un mail
PS: Our team reviews applications daily. Please complete your AI interview and application steps to be considered for this opportunity.
L'offre d'emploi a été publiée Il y a 2 mois
Des emplois similaires qui pourraient vous intéresserBasé sur l'offre DevOps Engineer - AI Model Evaluator à Toronto, ON
$85 par heure
...creative and technical talent with leading AI research labs. Headquartered in San... ..., and Jack Dorsey . Position: DevOps / SRE / Cloud Engineer (Coding Agent Experience) Type:... ...AI coding agents to complete and evaluate complex infrastructure engineering tasks...SuggéréTravail à distanceСontratTravail d'été$80 par heure
...and technical talent with leading AI research labs. Headquartered in... ...Jack Dorsey . Position: Data Engineer (Coding Agent Experience)... ...AI coding agents to complete and evaluate complex data engineering tasks. Review model-generated implementations involving...SuggéréTravail à distanceСontratTravail d'été- ...Palona’s AI agents operate in real restaurant environments: noisy phone lines... ...more than selecting the newest model. It requires disciplined evaluation, high-quality data, modeling judgment... ...looking for an applied AI Modeling Engineer to improve the intelligence, accuracy...SuggéréContrat Longue DuréeTemps pleinTravail temporaireStageRecrutement immédiat
$105k - $175k par année
...Job Type: Permanent Work Model: Hybrid Reference code: 135131 Primary Location... ...Summary We are hiring a Senior DevOps Engineer to own and evolve our cloud platform on AWS... ...like? You will lead how we adopt AI for infrastructure and platform work—not...SuggéréContrat Longue DuréeEmploi permanentHoraires flexibles$90k - $115k par année
...greener and smarter mobility worldwide, connecting cities as we reduce carbon and replace cars. Could you be the full-time DevOps Engineer in Toronto, ON we’re looking for? Your future role We’re looking for a motivated DevOps Engineer to help modernize and...SuggéréContrat Longue DuréeTemps pleinStageLe monde entier- ...looking for mission-driven talent to help accelerate our growth. The Opportunity SWTCH is looking for a highly motivated DevOps Engineer to join our growing tech team. Reporting to the Director of Engineering, you will design, build, and optimize our AWS container platform...Temps plein
$92k - $118k par année
...building secure, automated, and scalable DevOps solutions. Join Capco to help transform enterprise... ...data and analytics through cloud-native engineering and continuous delivery. The Role We... ...At Capco, we use artificial intelligence (AI) tools to support and enhance several...Temps pleinStageRecrutement immédiat$80k - $130k par année
Senior DevOps Engineering Consultant - Azure & AI Position Description The Senior DevOps Engineering consultant... ...and Kubernetes based deployment models. .Partner with engineering,... ...peuvent être utilisés pour aider à l’évaluation des candidatures, sans jamais remplacer...$140k - $175k par année
...Job Responsibility: About the team We are looking for brilliant staff DevOps engineers who have a passion for consumer products and want to join an energetic team that is changing the way people think about travel. We ship code daily, A/B test extensively, and have a primary...Temps pleinTravail au bureauZone localeTravail à distance- ...AGENT DEVOPS PLATFORM ENGINEER ROLE DESCRIPTIONS Design and operate AI agents that automate tasks across the software development lifecycle. Build and maintain... ...identity governance, and least-privilege access models. • Embed security, compliance, vulnerability...Emploi permanent
- ...Job Responsibility: We are looking for "Senior AWS DevOps Engineer" in Canada. Full-time Permanent/T4 (hourly) - 100% Remote Responsibilities:- By utilizing Python, NodeJs, CI/CD, GitOps Methodologies, etc., create and extend Full Stack infrastructure automation for enterprise...Taux horaireEmploi permanentTemps pleinTravail à distance
$72.06k - $138.94k par année
...At Capgemini Engineering, the world leader in engineering services, we... ...seeking a highly skilled Senior DevOps Engineer to help build, operate... ...through automation and AI-assisted operations. Your role... ...Understanding of ML lifecycle management, model deployment, and production...Emploi permanentTemps pleinZone localeTravail à distance$157.4k - $179.6k par année
...Toronto, Ontario,Manager, Credit Modeling Strategy & AnalyticsAbout... ...who come together to build our models and policies which lead to our... ...Finance, Statistics, Science, Engineering)\n \n At least 1 year of experience... ...of artificial intelligence (AI) to enhance the candidate...Contrat Longue DuréeTravail au bureauTravail à domicileCapital One - CA
Toronto, ONil y a 28 jours- ...platforms such as Amazon EKS and Google Kubernetes Engine (GKE). ~ Provision and manage cloud... ...and tooling using Python to support DevOps processes. ~ Monitor system performance,... ...ZoomInfo (NASDAQ: GTM), the all-in-one AI GTM platform, enables sales, marketing, and...
$100 - $150 par heure
...technical talent with leading AI research labs. Headquartered in... ...data analyses , statistical modeling work , machine learning... ..., and technical reports. Evaluate AI-generated or human-created... ...detailed written justifications for evaluations and scores to maintain...Travail à distanceСontratTravail d'été- ...CoreFactor is searching for a DevOps Engineer on a permanent/full-time basis. This position is hybrid and will require the successful incumbent... ...four (4) times per week. We are seeking an experienced AI/ML DevOps Engineer to join our clients Data, AI & Analytics team...Emploi permanentTemps pleinTravail au bureau
$140.6k - $190.6k par année
...Job Description: Engineering Manager, Are you passionate about transforming... ...future of legal research with AI? Join our Global Westlaw... ...(RAG), and large language models (LLMs). You will also champion... ...review processes. Champion a DevOps-first mindset in which...Temps pleinStageTravail au bureauZone localeLe monde entierHoraires flexiblesTravail posté- ...Our team is looking for a DevOps engineer with at least 6 years cloud and Kubernetes experience including specialization in ArgoCD. Our project involves building automations for a greenfield managed service product in AWS and we are looking for candidates who are interested...Temps plein
- ...Director, Engineering, AI Job Description Are you passionate about transforming how legal professionals... ...to a mission-based operating model. Replace project-by-project handoffs with... ...retrieval and model integration through evaluation, deployment, and monitoring. Set the...Contrat Longue DuréeTemps pleinСontratTravail au bureauZone localeHoraires flexiblesTravail posté
- ...Number : R2868511 Position title : AI Engineer Department: Commercial Data Science... ...enterprise data foundation strategies Monitor, evaluate, and continuously improve AI application... ...emerging AI technologies, foundational models, frameworks, and engineering practices,...Travail au bureauTravail à domicileBureau à domicileHoraires flexibles
$109k - $173k par année
...City: Toronto Project Objectives & Role Summary The AI DevOps Specialist is primarily responsible for the technical... ...Celestica’s global HPS Artificial Intelligence (AI) sandbox, modeling, and software engineering toolchains. This specialist will bridge the gap...Travail temporaireZone localeTravail à distance- ...DevOps Engineer About us TitanFile's mission is to democratize secure client communication. This is a challenge worthy of pursuit! The costs... ...problems. We expect our engineers to work fluently with AI tooling and to apply the same engineering judgment to what it produces...Temps pleinTravail au bureauHoraires flexibles
$84k - $175k par année
...Job Type: Permanent Work Model: Hybrid Reference code: 133810... ...design and implementation of AI-powered applications and... ...solutions including: o Prompt engineering o Retrieval-Augmented... ...mitigation o Monitoring and evaluation of model outputs • Integrate...Emploi permanentHoraires flexibles- ...staging, and production. Working with developers, architects, testers, cybersecurity specialists, and operational teams, the Senior DevOps Engineer improves deployment consistency, reduces manual effort, strengthens traceability, and supports secure and repeatable releases....Contrat Longue DuréeСontratStage
- ...Our team is looking for a DevOps engineer with at least 7 years cloud experience including specialization with Identity Our project involves building automations for a greenfield managed service product in AWS and we are looking for candidates who are interested in designing...Temps plein
- ...We are seeking a Lead Azure DevOps Engineer with deep expertise in Azure Networking to drive the design, implementation, and optimization of... ...conform to best practices for Terraform IaC creation; a generic AI-developed Terraform solution is not enough. Ability to set the right...Travail à distance
$100 par heure
...Mindrift connects specialists with project-based AI opportunities for leading tech companies, focused on testing, evaluating, and improving AI systems. Participation is... ...evaluate clinical EHR vignettes for a medical AI evaluation program. General and internal medicine are our...Taux horaireEmploi permanentTemps partiel10 heures semaine- ...Senior AI Engineer Toronto Downtown, ON - Hybrid (4 Days WFO) In Person Interview Required... ...LLM pipelines, Agentic workflows, Model Context Protocol (MCP), Graph/RAG architectures... ...using Apache Airflow/DAGs. • Cloud & DevOps (AWS): Comprehensive knowledge of AWS infrastructure...Contrat Longue DuréeEmploi permanent
- ...Cloud Platform Engineer – AWS, Python, DevOps & AI/LLM Required Skillsets: • 3–5+ years of hands-on CI/CD, cloud, and platform engineering... ...: prompt management, tool use, agent architectures, evaluation, observability, and reliability. • Experience with Git...Emploi permanent
- ...Tiger Analytics is a global leader in AI and analytics, helping Fortune 1000... ...looking for a highly skilled AI Engineer with 7+ years of experience in software... ...AI/LLM APIs, AWS Bedrock, and other model providers. Infrastructure & DevOps: Experience with DevOps, CI/CD...Contrat Longue DuréeZone locale
Voulez-vous recevoir plus d'offres d'emploi ?
S'abonner et recevoir des offres d'emploi similaires à DevOps Engineer - AI Model Evaluator. Soyez parmi les premiers à postuler !
Recherches connexes