DevOps Engineer - AI Model Evaluator
$85 par heureMercor
Toronto, ON
- Travail à distance
About the job
Mercor connects elite creative and technical talent with leading AI research labs. Headquartered in San Francisco, our investors include Benchmark , General Catalyst , Peter Thiel , Adam D'Angelo , Larry Summers , and Jack Dorsey .
Position: DevOps / SRE / Cloud Engineer (Coding Agent Experience)
Type: Contract
Compensation: $85/hour
Location: Remote
Role Responsibilities
- Use frontier AI coding agents to complete and evaluate complex infrastructure engineering tasks.
- Review model-generated implementations involving cloud platforms , Kubernetes , CI/CD systems , and infrastructure automation .
- Identify bugs, edge cases, reliability issues, and failure modes in model outputs.
- Compare outputs from multiple frontier models to assess strengths and weaknesses.
- Apply professional engineering judgment to realistic infrastructure engineering scenarios.
Qualifications
Must-Have
- 2+ years of professional DevOps , SRE , or Cloud Engineering experience.
- Experience with AWS , Azure , GCP , Kubernetes , Terraform , CI/CD pipelines , or observability tooling.
- Regular use of AI coding agents like Cursor , Claude Code , Codex , Windsurf , Gemini CLI , or similar tools.
- Ability to evaluate model-generated infrastructure and reliability engineering solutions.
Preferred
- Experience supporting production-scale systems .
Compensation & Legal
- $400 per accepted task .
- Compensation is tied to accepted work.
Application Process (Takes 20–30 mins to complete)
- Upload resume
- AI interview based on your resume
- Submit form
Resources & Support
- For details about the interview process and platform information, please check:
- For any help or support, reach out to: Montrer un mail
PS: Our team reviews applications daily. Please complete your AI interview and application steps to be considered for this opportunity.
L'offre d'emploi a été publiée il y a 29 jours
Des emplois similaires qui pourraient vous intéresserBasé sur l'offre DevOps Engineer - AI Model Evaluator à Toronto, ON
$85 par heure
...and technical talent with leading AI research labs. Headquartered in San... ...Dorsey . Position: iOS Engineer (Coding Agent Experience) Type... ...AI coding agents to complete and evaluate complex engineering tasks. Review model-generated mobile application code...SuggéréTravail à distanceСontratTravail d'été$80k - $180k par année
...Zafin is an AI platform company helping regulated institutions modernize how critical... ...scale. What’s the Opportunity? The AI Evaluation Engineer ensures AI agent solutions are accurate,... ...of AI evaluation, large language model behaviour, reasoning quality, hallucination...SuggéréTemps plein$130k - $145k par année
...role We are seeking a highly skilled and passionate Senior DevOps Engineer, AI infrastructure to join our team! You will play a key role in... ...and refine advanced prompting strategies for LLMs. Test, evaluate, and analyze the performance of LLM and other GenAI models....SuggéréTemps pleinStageTravail au bureauHoraires flexibles$101k - $169k par année
...Job Type: Permanent Work Model: Hybrid Reference code: 131871 Primary Location... ...complex, high-impact engagements in the AI Model Risk Space, ensuring alignment with... ...Generative AI offerings, including testing, evaluating and benchmarking different LLMs and toolchains...SuggéréEmploi permanentСontratHoraires flexibles- ...Our team is looking for a DevOps engineer with at least 6 years cloud and Kubernetes experience including specialization in ArgoCD. Our project involves building automations for a greenfield managed service product in AWS and we are looking for candidates who are interested...SuggéréTemps plein
$90k - $130k par année
...space to grow. About the Role We are currently looking for DevOps Specialists for various large-scale projects. Why Join Levio?... ...on the Use of Artificial Intelligence in Recruitment We use AI enabled tools to help sort and review applications based on job related...Emploi permanentTemps pleinHoraires flexibles- ...Tiger Analytics is a global leader in AI and analytics, helping Fortune 1000... ...looking for a highly skilled AI Engineer with 7+ years of experience in software... ...AI/LLM APIs, AWS Bedrock, and other model providers. Infrastructure & DevOps: Experience with DevOps, CI/CD...Contrat Longue DuréeZone locale
$72k - $138k par année
...Job Type: Permanent Work Model: Hybrid Reference code: 130069 Primary... ...-- We are looking for a passionate AI Research Engineer to join our team. You will work at the... ...research and product engineering—designing, evaluating, and deploying generative AI (GenAI) systems...Emploi permanentHoraires flexibles$80k - $120k par année
...Zafin is an AI platform company helping regulated institutions modernize how critical work is designed, governed, and delivered. Our technology enables organizations to move faster while maintaining the governance, accountability, and control required in highly regulated environments...Temps pleinStageZone locale- ...inclusive world. Job Description We are seeking to hire a Senior DevOps Engineer for the Client's Branch DevOps team. The Branch DevOps Team is... ...) to troubleshoot and remediate pipeline issues. · Leverage AI-assisted development tools (GitHub Copilot) to accelerate...Emploi permanentTemps pleinZone localeTravail posté
- ...Our team is looking for a DevOps engineer with at least 7 years cloud experience including specialization with Identity Our project involves building automations for a greenfield managed service product in AWS and we are looking for candidates who are interested in designing...Temps plein
- ...platforms such as Amazon EKS and Google Kubernetes Engine (GKE). ~ Provision and manage cloud... ...and tooling using Python to support DevOps processes. ~ Monitor system performance,... ...that empowers businesses to grow faster with AI-ready insights, trusted data, and advanced...Le monde entier
$92k - $118k par année
...building secure, automated, and scalable DevOps solutions. Join Capco to help transform enterprise... ...data and analytics through cloud-native engineering and continuous delivery. The Role We... ...At Capco, we use artificial intelligence (AI) tools to support and enhance several...StageRecrutement immédiat$135k - $170k par année
...General Information: Job Title: AI Engineer Location: Toronto, ON (Onsite/Hybrid)... ...critical AI features Define and implement evaluation frameworks Optimize systems for... ...Thrive We support remote and hybrid work models by offering multiple office locations to...Contrat Longue DuréeTemps pleinStageTravail au bureauRecrutement immédiatTravail à distanceHoraires flexibles- ...organizations to solve complex business challenges through cloud, AI, data engineering, and modern software solutions. We foster a collaborative,... ...outcomes. The Opportunity We're looking for a Senior DevOps Engineer who is deeply hands-on and ready to lead from the front...Temps plein
- ...Job Responsibility: Sr DevOps Engineer At Cognizant, our global community sets us apartan energetic, collaborative, and inclusive workplace... ...for software development and delivery. Research and evaluate new technologies and tools to improve software development and...Temps pleinTravail au bureau2 jours par semaine
$80 - $120 par heure
...creative and technical talent with leading AI research labs. Headquartered in San... ...Position: Software / AI / IT / data Evaluator Type: Contract Compensation... ...structured written feedback to improve AI model outputs . Apply deep subject-matter...Travail à distanceСontratTravail d'étéTravail au bureau$92k - $118k par année
...Capco is seeking a DevOps/DevSecOps Engineer to join our Technology & Engineering practice in Toronto. This role is ideal for someone with a strong... ...Talent Acquisition At Capco, we use artificial intelligence (AI) tools to support and enhance several parts of talent...StageRecrutement immédiat$80k - $138k par année
...Job Type: Permanent Work Model: Hybrid Reference code: 133554 Primary Location... ...on the job coaching -- As a Data & AI Platform Engineer, you operate at the foundation of modern... ..., and upgrades using tools like Azure DevOps, GitHub Actions, or Jenkins. • Deploy,...Emploi permanentHoraires flexibles$80 - $120 par heure
...creative and technical talent with leading AI research labs. Headquartered in San... ...regulatory response with financial-services AI Evaluator Type: Contract Compensation... ...to meet deadlines while improving AI model performance . Qualifications Must...Travail à distanceСontratTravail d'étéTravail au bureau$135k - $150k par année
...at . The Opportunity: We’re looking for an AI Engineer, AI Platform & ML Engineering to join our Data & AI... ...on the platform patterns, MLOps practices, model lifecycle, deployment standards, monitoring, evaluation, and governance integration needed to move AI and...Temps pleinStage- ...Generative AI Engineer About Apertera Apertera is leading the evolution of language solutions... ...Generative AI (GenAI) and Large Language Model (LLM) technologies. You will report to... ...translation, cloud infrastructure, and evaluation. You'll play a pivotal role in pushing the...Temps pleinTravail posté
$70 par heure
...creative and technical talent with leading AI research labs. Headquartered in San... ...Remote Role Responsibilities Evaluate AI-generated responses to ensure accuracy... ...structured written feedback to improve AI model outputs . Identify nuances, implicit...Travail à distanceСontratTravail d'été$101k - $169k par année
...Job Type: Permanent Work Model: Hybrid Reference code: 133157 Primary Location: Toronto, ON All Available Locations: Toronto... ...required. Hands-on experience in utilizing data analytics and AI-ML (artificial intelligence-machine learning) methodologies to...Emploi permanentTravail manuelHoraires flexibles- ...Senior DevOps Engineer Location: Toronto Role Description We are looking for a driven, top-notch Senior DevOps Engineer who is... ...certification (AWS, Kubernetes, or similar) Experience using AI-assisted development tools Knowledge of: JWT OAuth 2.0...
$55 - $65 par heure
...Job Responsibility: Role: Salesforce DevOps Engineer Location: Toronto/Calgary- Hybrid Hire Type: Contract Job Summary: We are seeking a skilled Salesforce DevOps Engineer to streamline and optimize our Salesforce development and release processes. You will be...Taux horaireTemps pleinСontratEmploi en cddLundi au vendredi$84k - $175k par année
...Job Type: Permanent Work Model: Hybrid Reference code: 133810... ...design and implementation of AI-powered applications and... ...solutions including: o Prompt engineering o Retrieval-Augmented... ...mitigation o Monitoring and evaluation of model outputs • Integrate...Emploi permanentHoraires flexibles$95k - $115k par année
...focus on stronger infrastructure, better automation, practical AI adoption, improved reliability, stronger security practices, and... ...level. About the Role Big Viking Games is hiring a Senior DevOps Engineer to help design, maintain, secure, and modernize the infrastructure...Contrat Longue DuréeTemps pleinTravail au bureau3 jours par semaine- ...Place to Work ®, five years in a row! As a Senior Modeling & Simulation (M&S) Engineer , you will develop models, tools, and simulations of next-generation space... ...prototype, and use new tools and technology (including AI) to learn, grow, and improve. Success in this role requires...Temps pleinTravail au bureauProgramme de réinstallation
- ...Threat Modeler / Security Architect – AI Security, Cloud & DevSecOps Must Have Technical/Functional Skills... ...programs. Secure Design & Engineering Enablement • Develop, maintain... ...Develop innovative attack techniques to evaluate protective designs and existing...Emploi permanent
Voulez-vous recevoir plus d'offres d'emploi ?
S'abonner et recevoir des offres d'emploi similaires à DevOps Engineer - AI Model Evaluator. Soyez parmi les premiers à postuler !
Recherches connexes