Ai Model Evaluation Engineer
AfterQuery Experts
Evaluate code and model outputs with AfterQuery, applying strong programming fundamentals in a fully remote, asynchronous engagement.This opportunity suits software engineers, computer scientists, and app or web developers with at least two years of experience. AfterQuery matches qualified contributors with projects spanning AI training, code review, evaluation, and original task creation. You first complete one assessment before receiving ongoing project matches.Key Responsibilities:
Evaluate model outputs for correctness and qualityReview software code across AI projectsComplete engineering tasks for model trainingCreate original tasks for evaluation projectsDocument work clearly and independentlyRequirements:
At least two years of software engineering experienceFull-time experience or substantive internships acceptedStrong programming fundamentals in one major languageReliable and detail-oriented working styleIndependent work capability in asynchronous projectsBring programming expertise and careful evaluation skills to AfterQuery projects supporting AI model quality and training.
#J-18808-Ljbffr
Evaluate model outputs for correctness and qualityReview software code across AI projectsComplete engineering tasks for model trainingCreate original tasks for evaluation projectsDocument work clearly and independentlyRequirements:
At least two years of software engineering experienceFull-time experience or substantive internships acceptedStrong programming fundamentals in one major languageReliable and detail-oriented working styleIndependent work capability in asynchronous projectsBring programming expertise and careful evaluation skills to AfterQuery projects supporting AI model quality and training.
#J-18808-Ljbffr
Vacancy posted 1 hour ago
Similar jobs that could be interesting for youBased on the Ai Model Evaluation Engineer in Winnipeg, MB vacancy
- Evaluate frontier AI models with electrical engineering judgment, circuit analysis expertise, and flexible remote work from your own computer.DataAnnotation seeks an experienced electrical engineer for independent contract work evaluating real engineering tasks. Apply circuit...SuggestedContract workFor contractorsRemote workFlexible hours
DataAnnotation
Winnipeg, MB1 hour ago - Evaluate AI models using consulting judgment, structured analysis, and clear written rationale from your own practice. Work remotely as an independent... ...responses and select the strongest recommendationBuild evaluation tasks, rubrics, and model answersRequirements:...SuggestedFor contractorsRemote work
DataAnnotation
Winnipeg, MB1 hour ago - We are looking for a Finance - AI Model Evaluator to help train AI models. You will measure the progress of these AI chatbots, evaluate their logic, and solve problems to improve the quality of each model. In this role you will need to hold an expert level of financial reasoning...SuggestedHourly payFor contractors
Sustainable World, Inc.
Winnipeg, MB1 hour ago - DataAnnotation is seeking an experienced Electrical Engineer to guide frontier AI models in evaluating real electrical engineering tasks. The role leverages your design judgment, debugging, and hardware experience to teach AI systems to reason about circuits, power stages, and...SuggestedInternshipRemote work
DataAnnotation
Winnipeg, MB1 hour ago - Evaluate advanced statistical queries and AI-generated solutions with expertise in statistical inference. Work remotely... .... Your work also supports AI model interpretability through documented... ...statistical inference expertise and careful evaluation skills to strengthen accurate,...SuggestedRemote work
Crossing Hurdles
Winnipeg, MB1 hour ago - Elevate AI advancement at DataAnnotation as an Electrical Engineer, focusing on real-world circuit and systems challenges. Apply... ...AI outputs, and ensuring the models align with professional standards... ...problems from practical experience • Evaluate AI model outputs against...
DataAnnotation
Winnipeg, MB1 hour ago - Evaluate frontier AI models using circuit analysis and electronics engineering expertise. Work remotely from an eligible country, if applicable.This Electrical Engineering AI Evaluator designs realistic engineering problems grounded in professional practice. You will assess...InternshipRemote work
MissionHires
Winnipeg, MB1 hour ago - Build and evaluate production-grade Rust services remotely from Canada, applying backend engineering expertise to AI training and software validation.This senior-level work centers on backend... ..., reproducible testing, and analytical evaluation to produce accurate solutions for AI...Work at officeRemote work
Jobgether
Winnipeg, MB1 hour ago - Shape reliable AI systems through retrieval, RAG, evaluation, and monitoring for an energy platform supporting complex portfolio decisions. Use Python or Node.Js daily.As a mid-to-senior Applied AI Engineer, you will improve how AI quality, cost, and latency are measured in...
Luxoft
Winnipeg, MB1 hour ago - Évalue et améliore des modèles d’intelligence artificielle avec Python, SQL et des métriques de performance à la Banque Nationale.Comme scientifique de données senior IA, tu travailleras dans une équipe multidisciplinaire spécialisée en automatisation à l’échelle de la Banque...
National Bank of Canada
Winnipeg, MB1 hour ago $140k - $175k per year
Job Description Thomson Reuters Labs is seeking a Lead Research Engineer to lead the design and delivery of production-ready software for artificial intelligence (AI) model evaluation and benchmarking. You will build scalable evaluation infrastructure that enables research...InternshipWork at officeLocal areaFlexible hoursThomson Reuters
Winnipeg, MB1 hour ago- ...Rootly is hiring an AI Quality Engineer to own the evaluation and optimization of its agentic AI features. You will design test scenarios, probe how AI agents reason, and work with engineering to close the loop on performance across incident workflows. You will partner...
Rootly
Winnipeg, MB1 hour ago - ...G2i Inc. is seeking a Senior Software Engineer, AI Training, to help evaluate the quality of AI-assisted coding interactions. This is a remote contract role... ...hours and weekly commitment. You will review how models think like engineers, assess explanations, and provide clear...Contract workRemote workFlexible hours
G2i Inc.
Winnipeg, MB1 hour ago - ...what it costs us to run. That means building the evaluation infrastructure that tells us whether a model change made our product better or worse, and... ...degrading quality.This is a measurement-first engineering role. Today we ship AI features and form opinions about quality from...Live InRemote work
Velixo
Winnipeg, MB1 hour ago - ...experienced Machine Learning Engineer to contribute to development... ...and deploying large language modeling systems at significant scale.... ...scientists to inform and improve our models and systems.Fine‑tune... ...integration to improve planning, evaluation, and knowledge retrieval.A...
Valence Labs
Winnipeg, MB1 hour ago - ...Miratech is looking for a skilled Senior AI Engineer with Python, LangGraph and LangChain to join... ..., monitoring, telemetry, and evaluation platforms. ~ Strong knowledge of software... ...Nice to Have Hands-on experience with Model Context Protocol (MCP) and Agent-to-Agent...Full timeInternshipRemote workRelocation
- ...building a global community of experts to evaluate, challenge and improve advanced AI systems. We are looking for... ..., LLMs, Mathematics, Science, Engineering, Linguistics, Cybersecurity, Psychology... ...concepts and technical outputs – Model reasoning and performance – Data...Hourly payFor contractorsRemote workFlexible hours
TELUS Digital AI Data Solutions
Winnipeg, MB1 hour ago - # Python Developer-AI EvaluationZettaMine Labs Pvt. Ltd. · , CABack to resultsPython... ...: We are hiring Senior Software Engineers - Python / TypeScript (USA) for a short-term AI Engineering and Evaluation project focused on improving AI models through high-quality training data,...Hourly payContract workTemporary workFreelanceImmediate startRemote work
Linuxassociation
Winnipeg, MB1 hour ago - ...remote contract role for a Python Senior Software Engineer, AI Training, based in Canada. You will evaluate AI-generated coding interactions, assess their quality... ...role emphasizes thoughtful critique, non-execution-based evaluation, and the ability to convey #J-18808-LjbffrContract workRemote work
Jobgether
Winnipeg, MB1 hour ago $120k - $140k per year
...Fusion(“MTF”) technology, an engineering-driven approach to fusion energy... ...Position Overview:The Systems Modeling Engineer works with a team of... ...requirements, and identify and evaluate mitigation optionsBuilding and... ...,Experience applying AI to accelerate systems modeling...Flexible hoursGeneral Fusion
Winnipeg, MB1 hour ago- Use management consulting judgment to evaluate AI models against business problems, with flexible remote project availability through DataAnnotation. Your analysis shapes better business tools.This opportunity suits experienced Management Consultants and Strategy Consultants...Remote workFlexible hours
DataAnnotation
Winnipeg, MB1 hour ago $129.4k - $179.4k per year
...ambiguous situations. You will support Global AI Model Validation and Governance’s mandate of... ...activities. When appropriate, evaluate whether AI models are reliable, fair, safe... ...Statistics, Biostatistics, Data Science, Engineering, Mathematics, or a related quantitative...Long term contractTemporary workLocal areaFlexible hoursManulife
Winnipeg, MB1 hour ago- Lead AI evaluation strategy from Canada, applying automated testing and model governance to improve agent reliability across enterprise automations.As Lead Product Manager... ..., and self-improvement featuresPartner with engineering and design on intuitive configuration...
Jobgether
Winnipeg, MB1 hour ago $70k - $150k per year
Application Deadline: 11/19/2026 Address: 33 Dundas Street West Job Family Group: Data Analytics & Reporting This is a hybrid role in Toronto Creates and implements an integrated roadmap for all aspects of data governance. Develops, maintains and monitors data...Part timeBMO
Winnipeg, MB1 hour ago- ...Jobgether in Canada-based partner search is for a Senior Software Engineer to evaluate AI coding interactions produced by tools like OpenAI Codex, Claude Code, and Cursor. This remote, contract-based engagement focuses on judging technical judgment, usefulness, and developer...Contract workRemote work
Jobgether
Winnipeg, MB1 hour ago - The AI Risk Management and Governance team oversees the development, deployment and use... ..., algorithmic and artificial intelligence models. Our goal is to maximize the value of data... ...and produce the required risk opinions Evaluate AI systems and vendor solutions that incorporate...InternshipRemote workFlexible hours
Desjardins
Winnipeg, MB1 hour ago - Evaluate AI model outputs using data science, statistics, machine learning, and quantitative reasoning expertise. This work is focused on rigorous technical review.As a Junior Data Scientist, you will assess model responses across data science, statistics, ML, and quantitative...
Great Value Hiring
Winnipeg, MB1 hour ago - Shape Agentic AI governance and validation at BMO Financial Group... ...AI Scientist sits in the Model Risk Management group, reporting... ...AI Model Validation. You will evaluate enterprise AI systems for conceptual... ...quantitative testing and evaluation for models and applicationsProvide...
BMO
Winnipeg, MB1 hour ago - Design fair hiring models at Amazon using LLMs, GenAI, and machine learning in a production-focused environment. Apply research... ...You will collaborate with Research Scientists and Software Engineers to integrate AI-driven products into Amazon’s hiring process. The work includes...
Amazon.com
Winnipeg, MB1 hour ago - ...based in San Francisco, is hiring a Software Engineering evaluator to create cutting-edge datasets for training and benchmarking large language models. You will curate code examples, provide... ..., Java, Rust, and Go, while evaluating AI-generated code for efficiency and reliability...Contract work
Turing Enterprises, Inc.
Winnipeg, MB1 hour ago
Do you want to receive more vacancies?
Subscribe and receive similar vacancies to Ai Model Evaluation Engineer. Be the first to apply!
