Senior Ai Coding Agent Evaluator
OpenTrain AI, Inc.
Evaluate AI-generated code remotely from Canada using software expertise, detailed rubrics, and strong written communication skills. Flexible contract work offers 20+ hours weekly.As a Senior Software Engineer AI Data contributor, you will assess coding agents through codebases, tickets, source materials, and generated outputs. Your evidence-based reviews help improve AI system behavior. Prior AI training experience is not required.Key Responsibilities:
Evaluate model outputs against project guidelines and rubricsReview codebases, tickets, source materials, and generated codeDocument evidence-based judgments clearly in writingExplain decisions to improve model behaviorMaintain consistent quality across repeated review tasksRequirements:
Relevant professional, academic, or language expertiseStrong software and written communication skillsCareful attention to detail during reviewsAbility to apply detailed instructions and rubrics consistentlyComfort working independently on remote contractor tasksApply software judgment, careful review, and clear writing to evaluate AI coding agents for ongoing remote contract work.
#J-18808-Ljbffr
Evaluate model outputs against project guidelines and rubricsReview codebases, tickets, source materials, and generated codeDocument evidence-based judgments clearly in writingExplain decisions to improve model behaviorMaintain consistent quality across repeated review tasksRequirements:
Relevant professional, academic, or language expertiseStrong software and written communication skillsCareful attention to detail during reviewsAbility to apply detailed instructions and rubrics consistentlyComfort working independently on remote contractor tasksApply software judgment, careful review, and clear writing to evaluate AI coding agents for ongoing remote contract work.
#J-18808-Ljbffr
Vacancy posted 47 minutes ago
Similar jobs that could be interesting for youBased on the Senior Ai Coding Agent Evaluator in Winnipeg, MB vacancy
- Evaluate AI-generated code at Turing using Python, TypeScript, or another production language. Work with coding agents across real-world repositories and assess technical quality.This independent contractor role suits experienced, hands-on software engineers with five or more...SuggestedFor contractors
Turing Enterprises, Inc.
Winnipeg, MB42 minutes ago - ...G2i Inc. is seeking a Senior Software Engineer, AI Training, to help evaluate the quality of AI-assisted coding interactions. This is a remote contract role with flexible hours and weekly commitment. You will review how models think like engineers, assess explanations,...SeniorContract workRemote workFlexible hours
G2i Inc.
Winnipeg, MB44 minutes ago - Evaluate AI-generated code remotely for Alignerr while applying software engineering, debugging, and code review expertise. Work flexibly on an hourly... ...practical experienceApply software engineering, code evaluation, and precise technical communication to help Alignerr improve...SuggestedHourly payContract workRemote work
Alignerr
Winnipeg, MB43 minutes ago - ...Francisco, is hiring a Software Engineering evaluator to create cutting-edge datasets for... ...benchmarking large language models. You will curate code examples, provide precise solutions, and... ...C++, Java, Rust, and Go, while evaluating AI-generated code for efficiency and...SuggestedContract work
Turing Enterprises, Inc.
Winnipeg, MB43 minutes ago - Bloomberg’s Engineering AI team in Toronto seeks a Senior Applied AI Engineer to build AI-driven search, discovery, and workflow features. You will work... ...research and production. You will design, train, and evaluate ML models and agentic systems, publish findings, and participate...Senior
Bloomberg
Winnipeg, MB43 minutes ago $97k - $160k per year
AI That Moves the World. Careers That Move You Forward. At Cerence... ...bon assistant. Il faut bâtir l’agent qui l’entoure — la couche qui... ...tout en conservant une base de code commune plutôt que des versions... ...automated testing, behavioral evaluation of non-deterministic systemsWorkflow...SeniorApprenticeshipLocal areaRemote workWorldwideCerence Inc.
Winnipeg, MB45 minutes ago- ...Senior Evaluation and Performance Analyst Department: Executive and Indigenous Affairs This Job Opening is limited to employees with the GNWT/WSCC in indeterminate and term positions. The Department of Executive and Indigenous Affairs works to provide overall management...SeniorLong term contract
- NVIDIA is seeking a Senior Engineer to drive performance modeling, evaluation, and post-training of large language models specialized in code generation, software engineering, and agentic coding tasks. You will work at the intersection of model training, inference optimization...Senior
2100 NVIDIA USA
Winnipeg, MB42 minutes ago $132.6k - $174k per year
...through automation, agentic automation, and AI. We partner with business and... ...enterprise scale.Core Responsibilities As a Senior Software Engineer – AI Agents, you will lead the design and... ...technology selections, conduct high-impact code reviews focusing on system design and...SeniorLong term contractPermanent employmentPart timeManual laborlululemon
Winnipeg, MB48 minutes ago- ...Miratech is looking for a skilled Senior AI Engineer with Python, LangGraph and LangChain... ...Responsibilities Develop and enhance AI agents and orchestration capabilities using... ...observability, monitoring, telemetry, and evaluation platforms. ~ Strong knowledge of software...SeniorFull timeInternshipRemote workRelocation
- Architect next-generation AI agents remotely in Canada, combining LLM expertise with planning, memory, tool calling, and autonomous execution.This Senior AI Agent Architect role focuses on reliable production systems and intelligent automation platforms. You will lead architecture...SeniorRemote work
Jobgether
Winnipeg, MB49 minutes ago - ...retrieval, and cloud infrastructure at Osedea. Help transform experimentation into reliable production software.This senior technical position focuses on AI-powered applications beyond proof-of-concept delivery. You will make architectural decisions balancing autonomy, predictability...Senior
Osedea
Winnipeg, MB44 minutes ago - Design AI agents and complex software systems at lululemon, using AWS Bedrock, Python, and enterprise... ...onsite work four days weekly.As a Senior Software Engineer, you will lead AI agent... ...-service feature implementationEstablish coding, testing, deployment, and integration...Senior
lululemon
Winnipeg, MB42 minutes ago - Build and review coding tasks for AI systems as a contractor Research Engineer, using programming expertise in a remote environment.The work centers... ...on code generation, software engineering, and AI model evaluation. You will apply competitive programming, algorithms, and data...For contractorsWork at officeRemote work
LGI Healthcare Solutions Santé Inc.
Winnipeg, MB42 minutes ago - Build production AI systems with Python and multi-agent workflows at Guidepoint’s hybrid Toronto office. Shape... ...global expert networks.As a hands-on Senior AI Engineer, you will move prototypes... ...production across agent design, evaluation, APIs, and cloud deployment. Guidepoint...SeniorWork at office
Guidepoint Global
Winnipeg, MB43 minutes ago - Build production agents for Float’s intelligent financial operating system using frontier models, evaluation, and context management in Toronto’s hybrid workplace.As a Senior Applied AI Engineer, you will help create the foundation for Float’s financial agent. The work spans...Senior
Float
Winnipeg, MB45 minutes ago - Build AI agents for legaltech with Google ADK, LLMs, and production delivery as a Senior AI Developer in hybrid Montreal. Work directly with founders.Abbeal seeks a Senior AI... ...architecture, software delivery, reliability evaluation, and technical leadership. You will work with...SeniorPermanent employmentDirect hire
Abbeal
Winnipeg, MB46 minutes ago $120k - $220k per year
Senior Founding Engineer - AI Agents Salary: $120k – $220k CAD 0.25% – 1.5% Job Location: Toronto Experience: 3 years of exp Job Type: Full... ...any unique tools as part of our platform Build the evaluation harness that keeps quality high as we ship fast. Ship...SeniorFull timeFor contractorsInternshipWork at officeRemote workWork from homeTyce
Winnipeg, MB44 minutes ago- Build production-ready AI agents in Canada, applying multi-agent systems expertise to streamline... ...and support decision-making.As a Senior AI Developer in Product Operations, you will... ...measurementDevelop reusable orchestration, memory, evaluation, and guardrail componentsEstablish...Senior
Jobgether
Winnipeg, MB42 minutes ago - ...UK,Canadaand New Zealand. THE PURPOSE OF YOUR ROLEAs Senior Product Manager – AI Agents, you will own and set the vision for Actionstep's agent roadmap... ...models (LLMs), multi-agent orchestration, tool use, and evaluating agent quality. Expertise in driving product...SeniorLong term contractTemporary workInternshipRemote workFlexible hours
Actionstep
Winnipeg, MB45 minutes ago - Define reliable AI agent testing at CrowdStrike with Python and distributed systems expertise. Build automation for a large-scale cloud... ...toolingMaintain eval suites, datasets, and AI scoring pipelinesEnforce CI evaluation gates before releasesDetect unreliable AI output and testing...Senior
CrowdStrike
Winnipeg, MB42 minutes ago - Shape AI agent products at Actionstep using product strategy and LLM expertise in a remote environment with travel required.As Senior Product Manager, AI Agents, you will own Actionstep's agent roadmap... ...orchestration, agent quality evaluation, and user-centered design.Key Responsibilities...SeniorRemote work
Actionstep
Winnipeg, MB45 minutes ago - # Python Developer-AI EvaluationZettaMine Labs Pvt. Ltd. · , CABack... ...Opportunity: We are hiring Senior Software Engineers - Python / TypeScript... ...short-term AI Engineering and Evaluation project focused on improving AI... ...large codebases, such as VS Code, is an advantage.• Ability to...SeniorHourly payContract workTemporary workFreelanceImmediate startRemote work
Linuxassociation
Winnipeg, MB43 minutes ago - Build AI-powered communication experiences with React and TypeScript at EvenUp, working hybrid from Toronto or San Francisco three days weekly.EvenUp seeks Senior and Mid Level Frontend Engineers for its Agents engineering pod within Growth & Agents. You will create user experiences...Senior
EvenUp
Winnipeg, MB42 minutes ago - Thomson Reuters Toronto is seeking a senior engineering leader to design and ship AI agent platforms and production-ready AI systems. You will drive architecture across multiple parallel initiatives, mentor senior engineers, and work with product leadership to translate strategy...Senior
Thomson Reuters
Winnipeg, MB43 minutes ago - Workday, Inc. in Vancouver, Canada is seeking a Senior AI Engineer for the Agent Factory team to drive end-to-end system design and product integration of intelligent agents within Workday Financials. You will be hands-on in designing, training, and integrating models, balancing...Senior
Workday, Inc.
Winnipeg, MB42 minutes ago - Lead AI agent security product strategy at Tigera in hybrid Vancouver, using Kubernetes and cloud-native networking expertise. Shape capabilities for production enterprise workloads.This senior product management role spans strategy through execution across Tigera Lynx and Calico...Senior
Tigera
Winnipeg, MB49 minutes ago - ...conversation into a competitive advantage. Cresta’s unified AI platform combines conversational AI agents, real‑time human agent augmentation, and comprehensive... ...and Golang, with the ability to write clean, efficient code.AI/ML Knowledge: Familiarity with AI/ML concepts. Hands‑...SeniorFull timeWork at officeLocal areaRemote workHome office
FDE Times
Winnipeg, MB43 minutes ago - Build reliable autonomous AI at Equifax through LLM evaluation, Python automation, and multi-agent workflow testing. This newly created position focuses on production reliability.As an Agentic AI Evaluation & Tuning Engineer, you will connect LLM capabilities with dependable...
Equifax Inc.
Winnipeg, MB43 minutes ago - Replicant in Canada is hiring a Senior Software Engineer, Agents on our Delivery team, to own the end-to-end development of AI agents for our customers. You’ll sit with clients to... ...tool calls, and help scale live conversations with guardrails and evaluation. #J-18808-LjbffrSenior
Replicant
Winnipeg, MB49 minutes ago
Do you want to receive more vacancies?
Subscribe and receive similar vacancies to Senior Ai Coding Agent Evaluator. Be the first to apply!
Related searches
- représentant senior Winnipeg, MB
- représentant alimentaire Winnipeg, MB
- agent d entretien Winnipeg, MB
- quality control representative Winnipeg, MB
- agent aux communications Winnipeg, MB
- agent de liaison Winnipeg, MB
- representant cosmetique Winnipeg, MB
- représentant industriel Winnipeg, MB
- représentant pharmaceutique Winnipeg, MB
- représentant interne Winnipeg, MB