AI Model Evaluation Engineer
AfterQuery Experts
Evaluate code and model outputs with AfterQuery, applying strong programming fundamentals in a fully remote, asynchronous engagement.
This opportunity suits software engineers, computer scientists, and app or web developers with at least two years of experience. AfterQuery matches qualified contributors with projects spanning AI training, code review, evaluation, and original task creation. You first complete one assessment before receiving ongoing project matches.
Key Responsibilities:
- Evaluate model outputs for correctness and quality
- Review software code across AI projects
- Complete engineering tasks for model training
- Create original tasks for evaluation projects
- Document work clearly and independently
Requirements:
- At least two years of software engineering experience
- Full-time experience or substantive internships accepted
- Strong programming fundamentals in one major language
- Reliable and detail-oriented working style
- Independent work capability in asynchronous projects
Bring programming expertise and careful evaluation skills to AfterQuery projects supporting AI model quality and training.
#J-18808-LjbffrVacancy posted 20 hours ago
Similar jobs that could be interesting for youBased on the AI Model Evaluation Engineer in Montréal, QC vacancy
- ...of defense for the use of various types of models and, accordingly, is in charge of model... ...Conduct independent quantitative reviews of the AI models used within BNP Paribas – CUSO IHC... ...desk and the sales team. Identify and evaluate model risks (including performance, bias,...SuggestedLong term contractWork at officeMonday to fridayFlexible hours3 days per week1 day per week
- ...Mindrift connects specialists with project-based AI opportunities for leading tech companies, focused on testing, evaluating, and improving AI systems. This is a task-based... ...clinical EHR vignettes for a medical AI evaluation project. General and internal medicine are the...SuggestedPart timeFor contractorsRemote workFlexible hours
- ...serve humanity. We’re training and deploying frontier models for developers and enterprises who are building AI systems to power magical experiences like content... ...our customers. Cohere is a team of researchers, engineers, designers, and more, who are passionate about their...SuggestedFull timeWork at officeRemote workFlexible hours
$180k - $230k per year
...complex capital projects. The AI buildout runs on physical... ...re looking for an Applied AI Engineer to help build the core ML engine... ..., train and deploy the models that perform Foresight's risk... ...~ Continuous improvement: Evaluate models repeatedly and prototype...SuggestedFull timeFor contractorsInternshipWork at office- ...challenge consensus. The Team: As AI tools drive immense value across... ...users to the most cost-effective models. The Role: We are looking for a Platform Engineer to help build and operationalize... ...: Configure and manage multi-engine API gateways, implementing...Suggested
- ...is a next-generation data and AI consulting firm dedicated to accelerating... ..., AI technologies, data engineering, and digital transformation.... ...pipeline feeding it, and the evaluations proving it works. This role... ...solutions Learns new tools and models fast, and shares what works...Full time
$200k per year
...specializing in data, analytics & AI consulting, dedicated to... ...analysts, data scientists, data engineers, and digital experts are all dedicated... ...classical machine learning models to LLM-powered applications —... ...— orchestration layers, evaluation frameworks, guardrails, and observability...Full timeManual labor- ...challenge consensus. The Team: As AI tools drive immense value across the firm,... ...Team owns this unified mission. While the engineering side of our team builds the platform rails... ...their workloads from expensive proprietary models (like Claude Opus) to cheaper mid-tier or...
- ...mathematical and statistical modeling skills and our qualitative assessment... ...to assist clients in evaluating and addressing risk. Our solutions... ...will partner closely with engineering to deliver auditable, reproducible... ...skills, and an appetite for AI-enabled development. The role...Minimum wageFull timeManual laborWork at officeLocal areaRemote workFlexible hours3 days per week1 day per week
- ...Miratech is looking for a skilled Senior AI Engineer with Python, LangGraph and LangChain to join... ..., monitoring, telemetry, and evaluation platforms. ~ Strong knowledge of software... ...Nice to Have Hands-on experience with Model Context Protocol (MCP) and Agent-to-Agent...Full timeInternshipRemote workRelocation
- ...Position Description: Work model: Hybrid, minimum 2 days onsite Based in Montreal, we have been helping Canadian entrepreneurs manage... ...programs. In this context, the PSC is creating a new Lead AI Platform Engineer position. The role will be responsible for structuring and evolving...Full time
$114.75k - $162k per year
...product information, semantic models, documentation, scripts, and operational... ...into reliable foundations for AI-enabled applications. As part... ...support multiple products, engineering workflows, and lines of... ...structure, contextualize, govern, and evaluate content from multiple sources,...Full timeTemporary workDirect hireFlexible hours- ...technically advanced, and commercially astute Forward Deployed AI Value Engineer to join our centralized AI Value Realization team. In this... ...management required to transition to AI-augmented translation models. 3. Continuous Optimization & Feedback Loop Adoption & Enhancement...Long term contractFull timeContract workInternshipManual labor
- ...training and deploying frontier models for developers and enterprises who are building AI systems to power magical experiences... ...is a team of researchers, engineers, designers, and more, who are passionate... ...a corner Own North's agentic evaluation framework, ensuring evals are...Full timeWork at officeRemote workFlexible hours
- ...Description: Department Overview The Model Validation (MV) group is a... ...The Artificial Intelligence (AI) Model Validation (MV) group... ...R&D in the GenAI/Agentic/LLM evaluation and testing areas. Lead a team... ..., machine learning / AI, engineering, statistics, mathematics, etc....Full timeTemporary workWork from home
- ...what it costs us to run. That means building the evaluation infrastructure that tells us whether a model change made our product better or worse, and... ...quality. This is a measurement-first engineering role. Today we ship AI features and form opinions about quality from...Full timeLive InRemote work
- ...that deliver healthcare. Our AI sensing platform enables existing... ...We're looking for an Applied AI Engineer to take our growing collection of foundation models and ML components from manually run... ...the model landscape cold, treats evaluation as a first-class engineering problem...
- ...Position Description: CGI Montréal is hiring a Senior AI/Automation Engineer to industrialize how data products are designed, built, and operated... ...current workflows and deliver AI/automation pipelines using AI models and the MCP framework. Integrate with the platform:...Full time
- ...breakthrough outcomes for their toughest AI challenges. We blend the power... .... As a Senior Lead AI Engineer , you are primarily... ...approaches based on the data or modelling problem at hand. As an AI Engineer... ...Experience designing and operating evaluation frameworks and observability...Full timeWork at officeLocal areaRemote workFlexible hours3 days per week1 day per week
- ...hiring a Senior Machine Learning Engineer Scientist to lead the... ...graph-based and transformer-based modeling systems, along with production... ...experimentation, reproducibility, evaluation, and deployment. Define and... ...for advancing structured-data AI and building platforms that...Long term contractPermanent employmentFull timeLocal areaWorldwideFlexible hours
- ...and help create the unknown! Job Description As a Team Lead - Modeling (Unreal) at Ubisoft Montreal, you will support the modeling team... ...deliver performant and visually consistent content in Unreal Engine. You’ll collaborate closely with art, level design, production,...Full time
- ...serve humanity. We’re training and deploying frontier models for developers and enterprises who are building AI systems to power magical experiences like content... ...our customers. Cohere is a team of researchers, engineers, designers, and more, who are passionate about their...Full timeWork at officeRemote workFlexible hours
$150k - $195k per year
...About You We are looking for a Principal Engineer to take architectural ownership of the agent... ...contributor role on the DevX and AI team, and the decisions you make here — which... ...Define the deploy, versioning, and rollback model for developer-published content — custom blocks...Local areaWorldwide- .... Nous développons « Scientist AI », un système d'IA de pointe conçu... ...au développement de nos modèles généraux Scientist AI et de garde... ...(LLM frontier ), nous devons évaluer de manière rigoureuse et indépendante... ...fiches de modèles/systèmes — model cards , les rapports...ApprenticeshipWork at officeFlexible hoursDay shift
- ...The Model Risk Management (MRM) team embedded within the Risk Management function in SG CIB oversees model risk management. MRM is responsible... ...SG Americas and global models devel: Conceptual Soundness: Evaluate soundness of model choices and assess the quality of model design...Full timeLocal area
$80k - $155k per year
...recrutement, des outils d’IA peuvent être utilisés pour aider à l’évaluation des candidatures, sans jamais remplacer le jugement humain.... ...de l’information (TI) et en management au monde. Software Engineer – AI-Assisted Development Position Description Senior · Develops...- ...work to be considered for the role. The Modeling Artist is responsible for creating high-... ...and technical robustness of the delivered models. They produce optimized assets that meet Rodeo... ...for progress. Explore new tools (e.g., AI, procedural modeling, photogrammetry) to...Permanent employmentFull time
$80k - $155k per year
...remediates vulnerabilities with the support of AI coding agents About our practice At CGI... ...security testing (DAST), threat modelling or application penetration testing. Knowledge... ...vacancy. #LI-NO1 Use of the term ‘engineering’ in this job posting refers to the technical...Full time$80k - $155k per year
...Senior · Develops and modernizes applications with the support of AI coding agents About our practice At CGI in Montréal, our Cloud... ...the facts and circumstances of each candidate. Use of the term ‘engineering’ in this job posting refers to the technical sense related to Information...Full timeusd218k per year
...technical leadership to team for design of multiple products Evaluates and recommends adoption of open source Evaluates third party... ...internal technical teams Software Reliability & Support Engineering Manages internal/external customer relationships and communication...Full timeWork at office
Do you want to receive more vacancies?
Subscribe and receive similar vacancies to AI Model Evaluation Engineer. Be the first to apply!
