AI Agent Evaluation Engineer - Remote
YO AI Labs
Edmonton, AB
- Remote job
Senior Software Engineer
Job Type: Contractor (~15 hours/week)
Location: Remote
Job Summary
We are seeking experienced Senior Software Engineers to support an AI training project by creating reinforcement learning environments that evaluate AI models on complex software engineering tasks using Model Context Protocol (MCP) tools.
You will design reproducible environments, deterministic verification, and reference solutions for tasks such as bug fixing, feature implementation, codebase refactoring, and performance optimization . No prior AI experience is required.
Key Responsibilities
- Create reinforcement learning environments for software engineering tasks.
- Design tasks involving bug fixing, feature development, refactoring, and performance optimization .
- Build deterministic verification systems and golden reference solutions.
- Evaluate AI agents' ability to reason through complex codebases and use MCP tools effectively.
- Develop realistic, reproducible software engineering scenarios.
- Ensure tasks accurately measure coding ability, problem-solving, and tool usage.
- Document solutions and provide clear technical feedback.
Required Skills
- Strong proficiency in Python 3, Java, Rust, C++, or TypeScript .
- Strong understanding of algorithms and data structures .
- Experience with bug fixing and debugging complex software issues .
- Proven experience in feature implementation and codebase refactoring .
- Strong knowledge of performance optimization and tuning .
- Excellent written and verbal communication.
- Strong attention to detail.
Preferred Qualifications
- Experience working with large-scale or distributed codebases .
- Familiarity with AI/ML systems is a plus but not required.
- Experience with rigorous code reviews and software engineering best practices .
- Experience working effectively in remote or cross-functional teams.
Hiring Process
- Submit an application and screening questions.
- Complete an AI interview (~30 minutes).
- Complete a technical assessment, if required.
- Hiring Manager review.
Compensation
Compensation is output-based , with payment provided per task that meets project specifications. Minimum weekly submission requirements may apply.
Availability
Selected experts should be prepared to begin their first tasks within 24–48 hours of completing onboarding .
Vacancy posted 5 days ago
Similar jobs that could be interesting for youBased on the AI Agent Evaluation Engineer - Remote in Edmonton, AB vacancy
- ...AI/ML Engineer Job Type: Full-Time Location: Remote Job Summary We are seeking an experienced AI/ML Engineer to build and deploy secure, scalable... ...work with LLMs, RAG, prompt engineering, multi-agent systems, and cloud AI platforms to develop production...Remote jobFull time
- ...Insurance Company’s distribution network, is expanding and seeking motivated individuals to join our team! About the Role: As a Remote Insurance Benefits Specialist , you’ll be the first point of contact for families seeking life insurance coverage. No prior insurance...Remote jobWeekly payFull timeContract workWork from homeLong distanceFlexible hours
- ...Flexible work schedule with hybrid work opportunities (office, home, or remote) * Competitive compensation program with performance recognition... ...Security Advisors. Your Role? As a Life insurance agent, you will help clients achieve their financial goals by providing...Remote workLong term contractWork at officeWork from homeFlexible hours
$89.71k - $112.17k per year
...we're searching for someone dedicated to making a difference to join our People & Talent Operations team as a Compensation and Job Evaluation Specialist. Reporting to the Director, People and Talent Operations, the Compensation and Job Evaluation Specialist is a subject matter...SuggestedPermanent employmentFull timeInternshipSecond jobImmediate startFlexible hours$16.5 per hour
...making their junk disappear. 1-800-GOT-JUNK?, part of O2E Brands, is hiring happy, hard-working individuals to join our 250+ Remote Inbound Sales Agents across Canada. Hiring Immediately for July 4th Full-Time - Work 5 days at 8.5 hours, between 10am - 8:30pm MDT Get...Remote workHourly payFull timeImmediate startWork from homeTrial periodMonday to friday$21.5 per hour
...Indigenous cultures, and bring vital services to some of the most remote and remarkable places in Canada. Why You’ll Love Working... ...Canadian airline About the position The Customer Service Agent is responsible for ensuring the highest level of customer service...Remote workHourly payLong term contractFull timeTemporary workWork at officeWorldwideTrial periodRelocation packageShift workDay shift- ...human-centered design, agile delivery, modern data practices, and AI-enabled approaches to improve service outcomes and advance... ...government. Working within multidisciplinary product teams, Data Engineer(s) will collaborate with business and technical stakeholders to understand...Remote jobFull timeFor contractors
- ...data continue to grow, creating a sustained need for dedicated data engineering and data product expertise. The Data Product Analyst role is... ...reliability, scalability, and performance across the platform. Utilize AI and automation tools to streamline data engineering workflows,...Remote jobShift work
- ...also committed to shaping the future of your career. As an AI and Analytics Analyst for PCL Constructors Inc. in Edmonton, Alberta... ...automation to optimize HR processes strategically. Identify, evaluate, and prioritize AI use cases that deliver measurable business...Long term contractFlexible hours
- ...fair and accessible while following all relevant human rights and privacy laws. We appreciate everyone who has shown interest in this position. Only those selected for an interview will be contacted. Please note, AI may be used in the selection and recruitment process....Full timeInternshipWork at office
- ...Full-time | Part-time | Commission-based Experience Level: New Agents, Experienced Agents, or Those Planning to Become Licensed Grow your... ...a business. Innovative Tech Ecosystem Work smarter with AI-powered rlpSPHERE’s built-in ecosystem with website, CRM, nurturing...Full timePart timeLocal areaShift work
- ...alongside iconic brand names. About the Role: As a luxury brand evaluator, you will step into the world of luxury to discreetly assess... ...in the luxury market. Compensation: • Non-Purchase Evaluations: Earn a fee based on mission complexity. • Purchase-Based Evaluations...Contract workWorldwideFlexible hours
- ...PCL opportunities. We are currently building enterprise-wide AI capability creating exciting opportunities to apply your skills... ...Staying informed on emerging AI technologies and contributing to the evaluation and recommendation of tools and platforms. Qualifications...Full time
$600 per week
...for motivated, people-oriented individuals who are ready to build a rewarding career in sales. In this full-time Door-to-Door Sales Agent role, you’ll connect directly with customers, represent trusted TELUS products, and be part of a high-energy, supportive team that...Daily paidFull timeFlexible hoursThe Acquisition Group
Edmonton, AB4 days ago$15 - $16 per hour
...Job Responsibility: Guest Service Agent Ramada by Wyndham Edmonton South About Us: At Ramada by Wyndham Edmonton South , our commitment goes beyond just providing rooms; it is about creating memorable experiences for every guest. Our dedication to exceptional service...Hourly payFull timePart timeMonday to fridayFlexible hoursShift workNight shiftDay shiftAfternoon shift- ...to site from a pickup point in Edmonton. The Guest Services Agent goes beyond checking in and out. You are responsible for acting... ...administrative experience is required Minimum of 6 months experience in a remote lodge setting Minimum of 1 year accounting experience that...Remote workFull timeInternshipShift work
$600 per week
...for motivated, people-oriented individuals who are ready to build a rewarding career in sales. In this full-time Door-to-Door Sales Agent role, you’ll connect directly with customers, represent trusted TELUS products, and be part of a high-energy, supportive team that...Daily paidFull timeFlexible hours- ...items sold in center). Provide back-up for our Member Services Coordinators. Various Admin tasks as assigned including assisting agents with hold support, document printing, inventory, gift card counts, custodian, cash out and other duties as assigned. Assess member...Full timeWork at officeLocal area
$21 per hour
Job Responsibility: WHO WE ARE: 310-DUMP rapidly expanding fast-paced Waste & Recycling Company. Established in 1995 , 310-DUMP is a very established Waste & Recycling Company providing services to residential, commercial and industrial customers. 310-DUMP is headquartered...Hourly payPermanent employmentFull timeCasual workMonday to fridayFlexible hoursShift workDay shiftAfternoon shift- ...Type: Contractor Location: Remote – United States Job... ...legal expertise to an innovative AI-focused project. The role involves... ...and reviewing legal content, evaluating AI-generated outputs, and... ...Familiarity with rubric-based evaluation of legal documents is a plus....Remote jobFor contractors
$3600 - $6000 per month
...the founder on technical solution design and development of the AI tabletop game product； Build and maintain a simple product showcase... ...: Bachelor's degree or above in Computer Science, Software Engineering, Human-Computer Interaction, Digital Media, or related fields preferred...Full timeInternshipRelocation- ...Job Responsibility: Overview We are seeking a motivated and dynamic Commercial Real Estate Agent to join our team. In this role, you will be responsible for facilitating the buying, selling, and leasing of commercial properties. You will leverage your knowledge of the Edmonton...Full timeFixed term contractFlexible hours
$16.2 - $19.35 per hour
...:00pm EST Department: Inbound Sales Location: Open to all agents located in Alberta, Manitoba, Ontario, New Brunswick, Prince Edward... ...posting is for future vacancies. Applications are screened with AI technology and reviewed by NTT DATA recruiters. #NBIND Inclusion...Remote workLong term contractFull timeTemporary workWork from homeMonday to fridayFlexible hoursAfternoon shift- ...all intake, routing, and coordination activities using established processes and systems, without reliance on artificial intelligence (AI) for clinical interpretation or decision-making. ~ Handle sensitive personal and medical information in accordance with privacy and...Permanent employmentFull timeTemporary workPart timeWork at officeWeekend workAfternoon shift
- “Do you want to make a meaningful impact in AI literacy? We are seeking several AI Literacy Specialists to support the development of high-quality AI literacy content. In this role, you will design accessible, engaging learning resources and assets tailored for diverse audiences...Permanent employmentFull timeCasual workWork at officeFlexible hoursShift work
- ...Criminal Lawyer Job Type: Contractor Location: Remote (United States Only) Job Overview We are seeking... ...technology. In this role, you will help improve next-generation AI systems by reviewing, evaluating, and refining AI-generated legal content related to...Remote jobFor contractors
$95k - $125k per year
...constructed. Entuitive is looking for a Senior Structural Engineer, based in our Edmonton office to be part of our... ...please check out our page. At Entuitive, we prioritize human-centric evaluation methods and do not utilize AI for screening or recruitment processes....Full timeContract workWork at officeFlexible hours- ...: Contractor Location: Global, Fully Remote Schedule: Flexible, ~15 hours/week... ...computational geometry experience to support an AI training project involving technical... ...Key Responsibilities Interpret engineering drawings and create accurate 3D CAD models...Remote jobFor contractorsFlexible hours
- ...proud to be a leading provider of Engineering, Procurement, Construction and... ..., studies, reports, technical evaluations, and project proposals.... ...on annual calendar ) Hybrid remote work program ( up to 52 days per... ...unsolicited agency resumes please. AI technology is utilized by...Remote workLong term contractFor contractorsSummer workInternshipWork at officeLocal areaImmediate startFlexible hours
- ...shaping the future of fleet readiness with AI-enabled, connected fleet technology that... ...Fleetworthy Inc. is hiring a Platform Engineer to own and evolve the systems that connect... ...VPN behavior, and TCP socket data flows in remote environments where recoverability matters...Remote work
Do you want to receive more vacancies?
Subscribe and receive similar vacancies to AI Agent Evaluation Engineer - Remote. Be the first to apply!
Related searches
- représentant développement des affaires Edmonton, AB
- parts representative Edmonton, AB
- security agent Edmonton, AB
- représentant senior Edmonton, AB
- représentant interne Edmonton, AB
- signing agent Edmonton, AB
- représentant représentante en assurances Edmonton, AB
- quality control representative Edmonton, AB
- de représentant Edmonton, AB
- agent aux communications Edmonton, AB