AI Agent Evaluation Engineer Outlier
Outlier
Evaluate AI agents for Outlier remotely, using backend engineering and system integration expertise to improve autonomous model behavior.
This freelance opportunity supports projects training large language models as proactive, multi-step agents. Contributors write evaluation rubrics, debug agent traces, and stress test systems for complex architectural workflows. The work suits engineers with two or more years in backend engineering, AI automation, or complex systems integration, including experience with production-grade software.
Key Responsibilities:
- Write rubrics evaluating agent performance and behavior
- Debug traces from autonomous agent workflows
- Stress test agents against complex scenarios
- Assess multi-step coordination and system interactions
Requirements:
- Two-plus years in backend engineering or AI automation
- Experience with modular production software architectures
- Practical experience using live, non-mocked environments
- Ability to deliver clear technical feedback
- Attention to subtle system failures
Help Outlier improve autonomous agents by combining AI evaluation, backend engineering, and complex systems integration experience.
#J-18808-LjbffrVacancy posted 16 hours ago
Similar jobs that could be interesting for youBased on the AI Agent Evaluation Engineer Outlier in Ottawa, ON vacancy
- ...Help Outlier improve autonomous agents using agent debugging, stress testing, and technical feedback through... ...This contributor opportunity supports AI organizations training large language... ...proactive, multi-step tasks. You will evaluate complex architectural workflows and identify...SuggestedFreelanceRemote work
Outlier
Ottawa, ON16 hours ago - ...Evaluate AI agents through rubrics, trace debugging, and stress testing while working remotely for Outlier. Use GPT, Claude, and other leading LLMs. Outlier helps innovative companies improve AI agents with detailed human feedback. This freelance engineering work supports...SuggestedFreelanceRemote work
Outlier
Ottawa, ON16 hours ago - ...Evaluate AI agents remotely for Outlier using technical feedback, trace debugging, and stress testing across real-world interactions. Outlier collaborates with AI organizations to train Large Language Models as proactive, multi-step agents. This freelance work centers on...SuggestedFreelanceWork at officeRemote work
Outlier
Ottawa, ON16 hours ago - ...Evaluate AI agents remotely through rubric writing, trace debugging, and stress testing. Outlier offers freelance work for experienced software engineers. This AI Agent Evaluation Engineer opportunity supports Outlier projects training Large Language Models as proactive, multi...SuggestedFreelanceRemote work
Outlier
Ottawa, ON16 hours ago - ...Evaluate AI agents for Outlier through rubric writing, trace debugging, and stress testing. Work remotely and freelance on a schedule you choose. Outlier helps innovative companies improve AI agents by supplying human feedback for large language model development. This opportunity...SuggestedFreelanceRemote work
Outlier
Ottawa, ON16 hours ago - ...Evaluate autonomous AI agents remotely using technical feedback, trace debugging, and stress testing for Outlier’s generative AI projects. Outlier collaborates with AI organizations to improve large language models through human feedback. This freelance work focuses on teaching...FreelanceRemote work
Outlier
Ottawa, ON16 hours ago - ...Evaluate advanced AI agents remotely with Outlier, applying technical feedback, debugging, and complex systems integration expertise. This freelance opportunity supports projects training large language models to function as proactive, multi-step agents. Contributors help...FreelanceRemote work
Outlier
Ottawa, ON16 hours ago - ...Evaluate autonomous AI agents through rubric writing, trace debugging, and stress testing while working remotely as a freelance contributor for Outlier. Outlier helps innovative companies improve AI agents by collecting human feedback. This work focuses on training large...FreelanceRemote work
Outlier
Ottawa, ON16 hours ago - ...Evaluate AI agents remotely using debugging, stress testing, and technical analysis for Outlier’s large language model projects. Outlier collaborates with AI organizations to improve autonomous systems through human feedback. This freelance work focuses on teaching agents...FreelanceRemote work
Outlier
Ottawa, ON16 hours ago - ...Apply AI automation and backend engineering skills to evaluate OpenClaw agents remotely through Outlier’s freelance contributor platform. Weekly payments are available. Outlier collaborates with leading AI organizations to improve large language models through human feedback...Weekly payFreelanceRemote work
Outlier
Ottawa, ON16 hours ago - ...Evaluate advanced AI agents remotely with Outlier, focusing on technical feedback, agent traces, and real-world system behavior. Work freelance. Outlier helps innovative companies improve AI agents through human feedback on large language model projects. This contract opportunity...Contract workFreelanceRemote work
Outlier
Ottawa, ON16 hours ago - ...Evaluate autonomous agents through rubric writing and trace debugging with Outlier, working remotely on flexible freelance projects. Outlier helps innovative companies improve AI agents through human feedback. This engineering work supports large language models designed...FreelanceRemote workFlexible hours
Outlier
Ottawa, ON16 hours ago - ...Evaluate AI agents with software engineering and technical feedback skills. Outlier offers remote freelance work for contributors in listed countries. This work supports leading AI organizations training large language models as proactive, multi-step agents. You will assess...FreelanceRemote work
Outlier
Ottawa, ON16 hours ago - ...Evaluate autonomous AI agents with Outlier using technical feedback, agent debugging, and complex systems integration. This freelance work is remote. Outlier helps innovative companies improve AI agents by collecting human feedback. As an engineer, you will help train large...FreelanceRemote work
Outlier
Ottawa, ON16 hours ago - ...Mindrift connects specialists with project-based AI opportunities for leading tech companies, focused on testing, evaluating, and improving AI systems. This is a task-based... ...clinical EHR vignettes for a medical AI evaluation project. General and internal medicine are the...Part timeFor contractorsRemote workFlexible hours
- ...Miratech is looking for a skilled Senior AI Engineer with Python, LangGraph and LangChain to... ...Responsibilities Develop and enhance AI agents and orchestration capabilities using... ..., monitoring, telemetry, and evaluation platforms. ~ Strong knowledge of software...Full timeInternshipRemote workRelocation
$104k - $154k per year
...Huawei Canada has an immediate 12-month contract opening for an Engineer. About the team: Welcome to the Advanced Wireless... ...solutions. About the job: Drive the evolution of multi-agent communication in AI networks through advanced solution development and system-level...Permanent employmentFull timeContract workInternshipImmediate startusd15 per hour
...online by using just a few words? Do you want to influence the future of global AI search? If so, we want you on board! Working on behalf of RWS, you will test and evaluate how our clients’ search engine responds to everyday searches from everyday people. Using your search...Weekly payFull timePart timeFreelanceSecond jobRemote workWork from homeHome officeFlexible hours$142.05k - $292k per year
...delivery role. It is not a research position. A Forward Deployed AI Engineer is a production engineer who works embedded inside a client's... ...prototyping, iteration, and deployment of AI-enabled workflows, agents, copilots, and automations into real operational environments...Full timeInternshipWork at officeLocal area$75k - $125k per year
...: We are seeking Intermediate Agentic AI Developers / AI Engineers to support an AI modernization initiative... ...to design, develop, and enhance AI agents and intelligent automation solutions using... ...orchestration capabilities. Evaluate, troubleshoot, and optimize AI agent accuracy...Full time- ...service à la clientèle sont essentiels - De bonnes aptitudes à la communication et aux relations interpersonnelles sont nécessaires pour évaluer les besoins des clients et formuler des recommandations - Une certaine expérience dans un rôle axé sur la clientèle est requise - La...
$85k - $111.3k per year
...tested, and maintained through Generative AI and agentic software engineering. We are seeking an Applied AI... ...use Large Language Models (LLMs), AI agents, knowledge systems, and software-analysis... ...use cases, prototype solutions, evaluate their impact on development efficiency...Long term contractFull timeTemporary workManual labor$69.45k - $119.45k per year
...business with Data & Advanced AI — backed by a $3B investment and... ...spanning LLM applications, agent orchestration, enterprise knowledge... ..., workflow automation, evaluation, observability and AI Ops. You... ...selection, prompt and context engineering, advanced RAG, tool/function calling...Full timeManual laborWork at officeLocal area- ...over 55 years. Backed by a legacy of engineering excellence, reliability and industry-leading... ...operations and end-to-end performance evaluation of the Telesat Lightspeed satellite... ...(IDEs). Familiarity with agentic AI, including AI agents and their use in engineering, software...Long term contractWorldwide
$69k - $86k per year
...software test procedure through Agentic AI-driven approach, such as autonomous agents, intelligent automation,... ...Sc in Telecommunication/Electrical Engineering /Computer Science or equivalent experience... ...AI engineering practice: agent evaluation, guardrails, observability, cost...Long term contractFull timeTemporary work- About Fullscript We’re an industry-leading health technology company on a mission to help people get better. We started in 2011 with one simple idea. Make it easier for practitioners to access the products they trust so they can deliver better care. That simple idea grew into...Full time
- ...Full-time | Part-time | Commission-based Experience Level: New Agents, Experienced Agents, or Those Planning to Become Licensed Grow your... ...a business. Innovative Tech Ecosystem Work smarter with AI-powered rlpSPHERE’s built-in ecosystem with website, CRM, nurturing...Full timePart timeLocal areaShift work
- .... From advancing healthcare and scientific discovery to powering AI and the technologies people rely on every day, innovation at AMD... ...career. THE ROLE: We are looking for a talented ASIC RTL Design Engineer to join our IP design team developing high-performance networking...Full timeInternship
$74.3k - $99.06k per year
...Nous recherchons un(e) Associé(e) principal(e), Impact ou d’un(e) agent(e), Climat et impact ! Rejoignez FinDev Canada, l’institution... ...L’équipe de l’Impact du développement est responsable de l’évaluation, de la mesure et du suivi des retombées sur le développement des...Permanent employmentFull timeContract workApprenticeshipWork at officeWorldwideDay shift- ...We are seeking a senior Professional Engineer who has built the majority of their career... ...and operational issues. ~ Develop and evaluate future growth, climate, operational, infrastructure... ..., scripting, automation tools, or AI agents to improve individual productivity and...Permanent employmentFull timeWork at office
Do you want to receive more vacancies?
Subscribe and receive similar vacancies to AI Agent Evaluation Engineer Outlier. Be the first to apply!
Related searches
- représentant développement des affaires Ottawa, ON
- enrolled agent Ottawa, ON
- représentant pharmaceutique Ottawa, ON
- de représentant Ottawa, ON
- représentant médical Ottawa, ON
- agent aux communications Ottawa, ON
- représentant interne Ottawa, ON
- security agent Ottawa, ON
- quality control representative Ottawa, ON
- vendeur représentant Ottawa, ON