Senior Ai Coding Agent Evaluator
OpenTrain AI, Inc.
Evaluate AI-generated code remotely from Canada using software expertise, detailed rubrics, and strong written communication skills. Flexible contract work offers 20+ hours weekly.As a Senior Software Engineer AI Data contributor, you will assess coding agents through codebases, tickets, source materials, and generated outputs. Your evidence-based reviews help improve AI system behavior. Prior AI training experience is not required.Key Responsibilities:
Evaluate model outputs against project guidelines and rubricsReview codebases, tickets, source materials, and generated codeDocument evidence-based judgments clearly in writingExplain decisions to improve model behaviorMaintain consistent quality across repeated review tasksRequirements:
Relevant professional, academic, or language expertiseStrong software and written communication skillsCareful attention to detail during reviewsAbility to apply detailed instructions and rubrics consistentlyComfort working independently on remote contractor tasksApply software judgment, careful review, and clear writing to evaluate AI coding agents for ongoing remote contract work.
#J-18808-Ljbffr
Evaluate model outputs against project guidelines and rubricsReview codebases, tickets, source materials, and generated codeDocument evidence-based judgments clearly in writingExplain decisions to improve model behaviorMaintain consistent quality across repeated review tasksRequirements:
Relevant professional, academic, or language expertiseStrong software and written communication skillsCareful attention to detail during reviewsAbility to apply detailed instructions and rubrics consistentlyComfort working independently on remote contractor tasksApply software judgment, careful review, and clear writing to evaluate AI coding agents for ongoing remote contract work.
#J-18808-Ljbffr
Vacancy posted 1 hour ago
Similar jobs that could be interesting for youBased on the Senior Ai Coding Agent Evaluator in Winnipeg, MB vacancy
- ...G2i Inc. is seeking a Senior Software Engineer, AI Training, to help evaluate the quality of AI-assisted coding interactions. This is a remote contract role with flexible hours and weekly commitment. You will review how models think like engineers, assess explanations,...SeniorContract workRemote workFlexible hours
G2i Inc.
Winnipeg, MB1 hour ago - Evaluate AI-generated code remotely for Alignerr while applying software engineering, debugging, and code review expertise. Work flexibly on an hourly... ...practical experienceApply software engineering, code evaluation, and precise technical communication to help Alignerr improve...SuggestedHourly payContract workRemote work
Alignerr Corp.
Winnipeg, MB1 hour ago - Use AI coding agents to deliver reliable applications at Remitly, with measurement, prompt iteration, and product ownership from requirements... ...directing AI agents to build and ship complete solutions. You will evaluate generated output against security, privacy, reliability, and...Suggested
Remitly
Winnipeg, MB1 hour ago - ...Francisco, is hiring a Software Engineering evaluator to create cutting-edge datasets for... ...benchmarking large language models. You will curate code examples, provide precise solutions, and... ...C++, Java, Rust, and Go, while evaluating AI-generated code for efficiency and...SuggestedContract work
Turing Enterprises, Inc.
Winnipeg, MB1 hour ago $90k - $140k per year
...Job Description Overview AtkinsRéalis is seeking a Senior Code Consultant – Fire Protection with 5–10+ years of experience to join our growing national fire protection practice. This is an opportunity to build your career on complex, meaningful infrastructure projects across...SeniorFull timeFor contractorsInternshipFlexible hours- Bloomberg’s Engineering AI team in Toronto seeks a Senior Applied AI Engineer to build AI-driven search, discovery, and workflow features. You will work... ...research and production. You will design, train, and evaluate ML models and agentic systems, publish findings, and participate...Senior
Bloomberg
Winnipeg, MB1 hour ago $100 per hour
...library. In this project, you will use an AI coding assistant to work on your existing... ...versions of AI-generated coding responses. Evaluate which response is more useful, accurate,... ...without using AI assistance to write evaluation responses. Nice to Haves: Experience...Hourly payContract workFreelanceRemote workWorldwide10 hours per weekLiftedanupworkcompany
Winnipeg, MB1 hour ago- ...Alignerr is seeking Software Engineers in Calgary for a fully remote, flexible contract role. You will build, evaluate, and improve AI-generated code, helping train AI systems to think like real developers. The role involves hands-on coding, code review, and providing expert...Hourly payContract workRemote workFlexible hours
Alignerr Corp.
Winnipeg, MB1 hour ago - Ayata seeks a Senior AI Engineer to design and build AI-powered solutions for understanding, transforming, migrating, and improving software systems and codebases. This hands-on role blends applied AI, software engineering, and cloud delivery. The ideal candidate has 5–10 years...Senior
E+E Consulting
Winnipeg, MB1 hour ago - ...Senior Evaluation and Performance Analyst Department: Executive and Indigenous Affairs This Job Opening is limited to employees with the GNWT/WSCC in indeterminate and term positions. The Department of Executive and Indigenous Affairs works to provide overall management...SeniorLong term contract
$97k - $160k per year
AI That Moves the World. Careers That Move You Forward .At Cerence... ...bon assistant. Il faut bâtir l’agent qui l’entoure — la couche qui... ...tout en conservant une base de code commune plutôt que des versions... ...automated testing, behavioural evaluation of non-deterministic systems...SeniorApprenticeshipLocal areaRemote workWorldwideCerence
Winnipeg, MB1 hour ago- We're hiring: AI Engineer | Downtown Toronto (Hybrid) | Full-Time... ...enterprise AI platform. It's hiring a Senior AI Engineer to join a small,... ...the business, with modern AI coding tools at the centre of how you... ...similar, including multi-step agent workflowsFull ownership of each...SeniorFull time
Encore Technical Solutions Inc.
Winnipeg, MB1 hour ago - NVIDIA is seeking a Senior Engineer to drive performance modeling, evaluation, and post-training of large language models specialized in code generation, software engineering, and agentic coding tasks. You will work at the intersection of model training, inference optimization...Senior
2100 NVIDIA USA
Winnipeg, MB1 hour ago - ...Novara is seeking a QA Engineer to own the quality bar for AI-powered features, including non-deterministic systems and agent-based workflows. You will define what getting it right means, build tooling, and partner with product and engineering to bring rigor to testing LLM-...Senior
Novara
Winnipeg, MB1 hour ago $132.6k - $174k per year
...through automation, agentic automation, and AI. We partner with business and... ...enterprise scale.Core Responsibilities As a Senior Software Engineer – AI Agents, you will lead the design and... ...technology selections, conduct high-impact code reviews focusing on system design and...SeniorLong term contractPermanent employmentPart timeManual laborLululemon Athletica Inc.
Winnipeg, MB1 hour ago- A leading technology firm in Canada is seeking a Senior Software Engineer / Coding Expert for a part-time, project-based contract. The role involves assessing and refining AI-generated code, designing unit tests, and providing technical feedback. Ideal candidates should have...SeniorContract workPart timeRemote workFlexible hours
Great Value Hiring
Winnipeg, MB1 hour ago - Use an AI coding assistant on your personally owned software project while comparing generated... ...code, improve documentation, and evaluate AI-generated coding responses. Terminal... ...software development, Git, debugging, and evaluation skills to improve your own project through...FreelanceRemote work
Liftedanupworkcompany
Winnipeg, MB1 hour ago - ...retrieval, and cloud infrastructure at Osedea. Help transform experimentation into reliable production software.This senior technical position focuses on AI-powered applications beyond proof-of-concept delivery. You will make architectural decisions balancing autonomy, predictability...Senior
Osedea
Winnipeg, MB1 hour ago - Design AI agents and complex software systems at lululemon, using AWS Bedrock, Python, and enterprise... ...onsite work four days weekly.As a Senior Software Engineer, you will lead AI agent... ...-service feature implementationEstablish coding, testing, deployment, and integration...Senior
Lululemon Athletica Inc.
Winnipeg, MB1 hour ago - # Python Developer-AI EvaluationZettaMine Labs Pvt. Ltd. · , CABack... ...Opportunity: We are hiring Senior Software Engineers - Python / TypeScript... ...short-term AI Engineering and Evaluation project focused on improving AI... ...large codebases, such as VS Code, is an advantage.• Ability to...SeniorHourly payContract workTemporary workFreelanceImmediate startRemote work
Linuxassociation
Winnipeg, MB1 hour ago - ...Miratech is looking for a skilled Senior AI Engineer with Python, LangGraph and LangChain... ...Responsibilities Develop and enhance AI agents and orchestration capabilities using... ...observability, monitoring, telemetry, and evaluation platforms. ~ Strong knowledge of software...SeniorFull timeInternshipRemote workRelocation
- Build production AI systems with Python and multi-agent workflows at Guidepoint’s hybrid Toronto office. Shape... ...global expert networks.As a hands-on Senior AI Engineer, you will move prototypes... ...production across agent design, evaluation, APIs, and cloud deployment. Guidepoint...SeniorWork at office
Guidepoint
Winnipeg, MB1 hour ago - Shape AI agent products at Actionstep using product strategy and LLM expertise in a remote environment with travel required.As Senior Product Manager, AI Agents, you will own Actionstep's agent roadmap... ...orchestration, agent quality evaluation, and user-centered design.Key Responsibilities...SeniorRemote work
Actionstep
Winnipeg, MB1 hour ago - Define reliable AI agent testing at CrowdStrike with Python and distributed systems expertise. Build automation for a large-scale cloud... ...toolingMaintain eval suites, datasets, and AI scoring pipelinesEnforce CI evaluation gates before releasesDetect unreliable AI output and testing...Senior
CrowdStrike Inc.
Winnipeg, MB1 hour ago - Build production-ready AI agents in Canada, applying multi-agent systems expertise to streamline... ...and support decision-making.As a Senior AI Developer in Product Operations, you will... ...measurementDevelop reusable orchestration, memory, evaluation, and guardrail componentsEstablish...Senior
Jobgether
Winnipeg, MB1 hour ago - ...UK,Canadaand New Zealand. THE PURPOSE OF YOUR ROLEAs Senior Product Manager – AI Agents, you will own and set the vision for Actionstep's agent roadmap... ...models (LLMs), multi-agent orchestration, tool use, and evaluating agent quality. Expertise in driving product...SeniorLong term contractTemporary workInternshipRemote workFlexible hours
Actionstep
Winnipeg, MB1 hour ago - Shape HouseSigma’s real estate platform and AI assistant remotely across Canada, combining design craft, coding, user research, and evidence.HouseSigma seeks a Product Builder who can understand users deeply and turn findings into working software. You may have developed through...Remote work
HouseSigma Inc.
Winnipeg, MB1 hour ago $120k - $220k per year
Senior Founding Engineer - AI Agents Salary: $120k – $220k CAD 0.25% – 1.5% Job Location: Toronto Experience: 3 years of exp Job Type: Full... ...any unique tools as part of our platform Build the evaluation harness that keeps quality high as we ship fast. Ship...SeniorFull timeFor contractorsInternshipWork at officeRemote workWork from homeTyce
Winnipeg, MB1 hour ago- Build reliable autonomous AI at Equifax through LLM evaluation, Python automation, and multi-agent workflow testing. This newly created position focuses on production reliability.As an Agentic AI Evaluation & Tuning Engineer, you will connect LLM capabilities with dependable...
Equifax Inc.
Winnipeg, MB1 hour ago - ...building a global community of experts to evaluate, challenge and improve advanced AI systems. We are looking for... ...– Data and technical accuracy – Coding and computational outputs LLM &... ...AI behaviour Domain-Specific AI Evaluation – Healthcare & Medicine – Law...Hourly payFor contractorsRemote workFlexible hours
TELUS Digital AI Data Solutions
Winnipeg, MB1 hour ago
Do you want to receive more vacancies?
Subscribe and receive similar vacancies to Senior Ai Coding Agent Evaluator. Be the first to apply!
Related searches
- représentant senior Winnipeg, MB
- représentant alimentaire Winnipeg, MB
- agent d entretien Winnipeg, MB
- quality control representative Winnipeg, MB
- agent nettoyage Winnipeg, MB
- agent aux communications Winnipeg, MB
- agent de liaison Winnipeg, MB
- representant cosmetique Winnipeg, MB
- représentant industriel Winnipeg, MB
- représentant pharmaceutique Winnipeg, MB