Ai Coding Model Evaluation Engineer
Turing Enterprises, Inc.
Evaluate AI-generated code at Turing using Python, TypeScript, or another production language. Work with coding agents across real-world repositories and assess technical quality.This independent contractor role suits experienced, hands-on software engineers with five or more years of experience. You will review agent behavior, compare solutions, identify failure modes, and create evaluation signals that improve coding models. The work includes collaboration with researchers and engineers.Key Responsibilities:
Evaluate AI-generated code across real-world software repositoriesReview agent behavior, tool usage, and code changesIdentify technical errors and recurring model failure modesCreate rubrics and evaluation criteria for coding tasksProduce preference data and actionable research findingsRequirements:
Five or more years of hands-on software engineering experienceStrong Python, TypeScript, JavaScript, Go, or equivalent proficiencyExperience working in substantial real-world codebasesStrong code-review skills and technical judgmentExperience using modern LLMs or AI coding toolsApply software engineering judgment, code review, and clear communication to improve Turing’s AI coding model evaluations.
#J-18808-Ljbffr
Evaluate AI-generated code across real-world software repositoriesReview agent behavior, tool usage, and code changesIdentify technical errors and recurring model failure modesCreate rubrics and evaluation criteria for coding tasksProduce preference data and actionable research findingsRequirements:
Five or more years of hands-on software engineering experienceStrong Python, TypeScript, JavaScript, Go, or equivalent proficiencyExperience working in substantial real-world codebasesStrong code-review skills and technical judgmentExperience using modern LLMs or AI coding toolsApply software engineering judgment, code review, and clear communication to improve Turing’s AI coding model evaluations.
#J-18808-Ljbffr
Vacancy posted 1 hour ago
Similar jobs that could be interesting for youBased on the Ai Coding Model Evaluation Engineer in Winnipeg, MB vacancy
- Evaluate AI-generated code remotely for Alignerr while applying software engineering, debugging, and code review expertise. Work flexibly on an hourly contract from Calgary.Alignerr seeks Software Engineers to build, evaluate, and improve AI-generated code. You will write solutions...SuggestedHourly payContract workRemote work
Alignerr
Winnipeg, MB1 hour ago - Evaluate frontier AI models with electrical engineering judgment, circuit analysis expertise, and flexible remote work from your own computer.DataAnnotation seeks an experienced electrical engineer for independent contract work evaluating real engineering tasks. Apply circuit...SuggestedContract workFor contractorsRemote workFlexible hours
DataAnnotation
Winnipeg, MB1 hour ago - Evaluate AI-generated code remotely from Canada using software expertise, detailed rubrics, and strong written communication skills. Flexible contract work offers 20+ hours weekly.As a Senior Software Engineer AI Data contributor, you will assess coding agents through codebases...SuggestedOngoing contractContract workFor contractorsRemote workFlexible hours
OpenTrain AI, Inc.
Winnipeg, MB1 hour ago - ...G2i Inc. is seeking a Senior Software Engineer, AI Training, to help evaluate the quality of AI-assisted coding interactions. This is a remote contract role with flexible hours and weekly commitment. You will review how models think like engineers, assess explanations,...SuggestedContract workRemote workFlexible hours
G2i Inc.
Winnipeg, MB1 hour ago - ...Flexspring is seeking an AI Model Training and Deployment Developer to join our engineering team building the next generation of our AI tools and iPaaS platform. You... ...will design, train, and deploy AI models using low-code AI studios and integrate AI workflows into HR data...SuggestedRemote workFlexible hours
Flexspring, Inc.
Winnipeg, MB1 hour ago - Turing, based in San Francisco, is hiring a Software Engineering evaluator to create cutting-edge datasets for training and benchmarking large language models. You will curate code examples, provide precise solutions, and correct code across Python, JavaScript (ReactJS), C/...Contract work
Turing
Winnipeg, MB1 hour ago - We are looking for a Finance - AI Model Evaluator to help train AI models. You will measure the progress of these AI chatbots, evaluate their logic, and solve problems to improve the quality of each model. In this role you will need to hold an expert level of financial reasoning...Hourly payFor contractors
Sustainable World, Inc.
Winnipeg, MB1 hour ago - Evaluate AI models using consulting judgment, structured analysis, and clear written rationale from your own practice. Work remotely as an independent... ...responses and select the strongest recommendationBuild evaluation tasks, rubrics, and model answersRequirements:...For contractorsRemote work
DataAnnotation
Winnipeg, MB1 hour ago - DataAnnotation is seeking an experienced Electrical Engineer to guide frontier AI models in evaluating real electrical engineering tasks. The role leverages your design judgment, debugging, and hardware experience to teach AI systems to reason about circuits, power stages, and...InternshipRemote work
DataAnnotation
Winnipeg, MB1 hour ago - NVIDIA is seeking a Senior Engineer to drive performance modeling, evaluation, and post-training of large language models specialized in code generation, software engineering, and agentic coding tasks. You will work at the intersection of model training, inference optimization...
2100 NVIDIA USA
Winnipeg, MB1 hour ago - Evaluate advanced statistical queries and AI-generated solutions with expertise in statistical inference. Work remotely... .... Your work also supports AI model interpretability through documented... ...statistical inference expertise and careful evaluation skills to strengthen accurate,...Remote work
Crossing Hurdles
Winnipeg, MB1 hour ago - Build and review coding tasks for AI systems as a contractor Research Engineer, using programming expertise in a remote environment.The work centers on code generation, software engineering, and AI model evaluation. You will apply competitive programming, algorithms, and data...For contractorsWork at officeRemote work
LGI Healthcare Solutions Santé Inc.
Winnipeg, MB1 hour ago - Elevate AI advancement at DataAnnotation as an Electrical Engineer, focusing on real-world circuit and systems challenges. Apply... ...AI outputs, and ensuring the models align with professional standards... ...problems from practical experience • Evaluate AI model outputs against...
DataAnnotation
Winnipeg, MB1 hour ago - Ayata seeks a Senior AI Engineer to design and build AI-powered solutions for understanding, transforming, migrating, and improving software systems and codebases. This hands-on role blends applied AI, software engineering, and cloud delivery. The ideal candidate has 5–10 years...
E+E Consulting
Winnipeg, MB1 hour ago - Build reliable autonomous AI at Equifax through LLM evaluation, Python automation, and multi-agent workflow testing. This newly created position focuses... ...reliability.As an Agentic AI Evaluation & Tuning Engineer, you will connect LLM capabilities with dependable autonomous...
Equifax Inc.
Winnipeg, MB1 hour ago - Flexspring, Inc. is seeking a skilled AI Model Training and Deployment Developer to join our engineering team, building the next generation of our AI tools and services... ..., training, and deploying AI models using low-code platforms and integrate HR applications within our...
Flexspring
Winnipeg, MB1 hour ago - Contribute your expertise in electrical engineering with DataAnnotation, evaluating AI models through real-world engineering challenges. Utilize your design and debugging skills to shape the future of AI technology. DataAnnotation is seeking an experienced Electrical Engineer...
DataAnnotation
Winnipeg, MB1 hour ago - We're hiring: AI Engineer | Downtown Toronto (Hybrid) | Full-Time A well-established global... ...teams across the business, with modern AI coding tools at the centre of how you work.AI-... ...Code, GitHub Copilot, Cursor and the latest models from Anthropic, OpenAI and Google...Full time
Encore Technical Solutions Inc.
Winnipeg, MB1 hour ago - Build and evaluate production-grade Rust services remotely from Canada, applying backend engineering expertise to AI training and software validation.This senior-level work centers on backend... ..., reproducible testing, and analytical evaluation to produce accurate solutions for AI...Work at officeRemote work
Jobgether
Winnipeg, MB1 hour ago - Evaluate AI-generated mechanical designs and calculations with strong engineering fundamentals and technical writing skills. Work fully remotely on flexible, project-based... ...or Python experienceBring careful engineering evaluation, failure analysis, and technical communication...For contractorsRemote workFlexible hours
Brunel
Winnipeg, MB1 hour ago - Shape reliable AI systems through retrieval, RAG, evaluation, and monitoring for an energy platform supporting complex portfolio decisions. Use Python or Node.Js daily.As a mid-to-senior Applied AI Engineer, you will improve how AI quality, cost, and latency are measured in...
Luxoft
Winnipeg, MB1 hour ago - Amazon in Vancouver, BC is seeking an AI Security Engineer to secure AI applications at scale and partner with software teams to build secure AI systems. You will drive security strategy, review designs and code, and develop automation to reduce risks. Join a diverse team focused...
Amazon Inc.
Winnipeg, MB1 hour ago - ...Coffee. Position Summary Join our AI team as an Intermediate AI Engineer, where you'll build and deploy AI... ..., deep learning, and generative AI models to solve specific business problems.... ...requirements into technical solutions. • Evaluate, monitor, and optimize models and...
$140k - $175k per year
Job Description Thomson Reuters Labs is seeking a Lead Research Engineer to lead the design and delivery of production-ready software for artificial intelligence (AI) model evaluation and benchmarking. You will build scalable evaluation infrastructure that enables research...InternshipWork at officeLocal areaFlexible hoursThomson Reuters
Winnipeg, MB1 hour ago- Bloomberg’s Engineering AI team in Toronto seeks a Senior Applied AI Engineer to build AI-driven search, discovery, and workflow features... ...research and production. You will design, train, and evaluate ML models and agentic systems, publish findings, and participate in open...
Bloomberg
Winnipeg, MB1 hour ago - Apple’s Creativity Apps team seeks a Machine Learning Engineer focused on evaluating ML models and AI features in our creativity tools. You will define evaluation metrics, build evaluation datasets, and develop scalable infrastructure to ensure quality from model development...
Lex
Winnipeg, MB1 hour ago - ...for leading and delivering key AI/Agentic initiatives under the... ...leverage their extensive software engineering experience in designing and... ...data products. Build evaluation harnesses and quality gates, including... ...practices, including code quality, automated testing, CI...Full timeFlexible hours
Aimlroles
Winnipeg, MB1 hour ago $80.9k - $129.73k per year
Product Engineering (Infrastructure as Code)This website uses cookies and similar technology to better understand... ..., and software-defined operating model. This role is open across Senior Consultant... ...automation technologies.* Apply AI-assisted engineering and Specification...Full timeInternshipWork at officeRemote workHome officeFlexible hoursAllstate Northern Ireland Limited
Winnipeg, MB1 hour ago- ...Rootly is hiring an AI Quality Engineer to own the evaluation and optimization of its agentic AI features. You will design test scenarios, probe how AI agents reason, and work with engineering to close the loop on performance across incident workflows. You will partner...
Rootly
Winnipeg, MB1 hour ago - ...Miratech is looking for a skilled Senior AI Engineer with Python, LangGraph and LangChain to join... ..., monitoring, telemetry, and evaluation platforms. ~ Strong knowledge of software... ...Nice to Have Hands-on experience with Model Context Protocol (MCP) and Agent-to-Agent...Full timeInternshipRemote workRelocation
Do you want to receive more vacancies?
Subscribe and receive similar vacancies to Ai Coding Model Evaluation Engineer. Be the first to apply!
