Ai Agent Reliability Engineer
TribalScale
Improve AI agent reliability at TribalScale with evaluation pipelines, quality metrics, and disciplined testing for agentic systems. The location remains to be confirmed.This Agentic QA Engineer role focuses on validating AI-driven workflows in real-world environments. You will design evaluation datasets, test scenarios, and benchmarks for dynamic application outputs. You will also collaborate with AI engineers, developers, product teams, and client-facing engineering teams.Key Responsibilities:
Develop pipelines measuring agent accuracy and task completionEstablish quality metrics, datasets, scenarios, and benchmarksEvaluate edge cases, recovery, security, and resilienceIntegrate quality checks into CI/CD pipelinesInvestigate production issues and improve test coverageRequirements:
Strong analytical thinking for complex real-world test casesExperience creating automated tests and testing frameworksStrong API, integration, regression, and end-to-end testing knowledgeExperience debugging applications and technical issuesFamiliarity with modern software development practicesBring quality engineering expertise to AI agent evaluation, production reliability, and measurable software performance at TribalScale.
#J-18808-Ljbffr
Develop pipelines measuring agent accuracy and task completionEstablish quality metrics, datasets, scenarios, and benchmarksEvaluate edge cases, recovery, security, and resilienceIntegrate quality checks into CI/CD pipelinesInvestigate production issues and improve test coverageRequirements:
Strong analytical thinking for complex real-world test casesExperience creating automated tests and testing frameworksStrong API, integration, regression, and end-to-end testing knowledgeExperience debugging applications and technical issuesFamiliarity with modern software development practicesBring quality engineering expertise to AI agent evaluation, production reliability, and measurable software performance at TribalScale.
#J-18808-Ljbffr
Vacancy posted 2 hours ago
Similar jobs that could be interesting for youBased on the Ai Agent Reliability Engineer in Winnipeg, MB vacancy
- Join Rootly as an AI Engineer specializing in developing reliable AI-driven solutions for incident management. Use your LLM experience to shape the future of business reliability. As a Senior AI Engineer at Rootly, you will lead the development of a transformative AI reliability...Suggested
$102k - $152k per year
...Equinix's GenAI products and platforms stable and reliable, and makes them more resilient over time.... ...a lasting fix. Works in a team with senior engineers and with product, platform and security partners to keep AI products, agents, connectors and self-hosted models robust,...SuggestedFull timeManual laborWork at office- Build enterprise AI agents at Decagon in an in-office role requiring technical delivery, integrations... ...-facing execution.As an Agent Deployment Engineer, you will own strategic customer... ...role connects customer requirements with reliable AI agent delivery.Key Responsibilities:...SuggestedWork at office
$130k - $155k per year
.... Our vision is to build an ML/AI first network of advertisers and... ...hiring its first dedicated AI Engineer. Over the past year, AI at Fluent... ...to production: a service desk agent that closes tickets without a... ...It is getting trusted data and reliable actions in front of it.You are...SuggestedWork at officeRemote work- Lead AI agent engineering at BioRender remotely, focusing on multi-agent orchestration, model routing, and dependable scientific figure generation... ...platform with engineering leadership, model evaluation, orchestration, and reliable scientific figure generation. #J-18808-LjbffrSuggestedRemote work
- Build AI agents for ClickStack using TypeScript, Node.Js, Python, and MCP in a product-focused engineering environment. Improve how developers investigate incidents.As an AI Product... ...experience, reusable skills, production reliability, and integrations. You will help agents...
- Shape AI agent platforms in Canada through tool calling, model routing, and production operations. Apply strong engineering judgment to practical systems.This Applied AI Engineer role centers... ...runtime architecture and orchestration for reliable customer workflows. You will design...
- As a Site Reliability Engineer II on the Serving Platforms team within Infrastructure Engineering, you will design, automate, and manage the core... ...RabbitMQ) in productionExperience operating, scaling, and monitoring AI/ML or LLM-powered services and workloads in high-concurrency...Worldwide
- Lead AI-powered reliability engineering at Questrade, using Python and machine learning to strengthen high-frequency trading platforms in a hybrid environment... ...work combines Google Cloud Platform, Google Kubernetes Engine, Terraform, Helm, observability, and intelligent...
- Design AI-agent workflows with Claude Code while building reliable engineering harnesses for software-development workflows in Toronto.This Agentic Engineer role focuses on introducing modern agentic engineering practices into a complex existing engineering organization. You...
- ...commitment to people. Our dependable quality, reliable delivery, and knowledgeable follow-up... ...and Coffee. Position Summary Join our AI team as an Intermediate AI Engineer, where you'll build and deploy AI applications, agents, and automation solutions that solve real business...
- Build production AI agents for ClickStack with TypeScript, Python, MCP, and a developer experience focus at ClickHouse. Work remotely across ClickHouse’s distributed company.As an AI Product Engineer, you will create agentic capabilities on ClickStack, ClickHouse’s open-source...Remote work
- Build AI agents for renewals and expansion at ShareGate, using low-code orchestration, Python or TypeScript, and SQL remotely across Canada... ..., marketing operations, customer success operations, or GTM engineering backgroundHands-on n8n, Make, Zapier, or similar experiencePython...Remote work
- Lead AI figure agent engineering at BioRender, focusing on multi-agent orchestration and dependable scientific figure generation. This remote-first... ..., planning, tool use, self-correction, and production reliability. The work combines frontier LLM and VLM evaluation, model...Remote work
- Workday, Inc. in Vancouver, Canada is seeking a Senior AI Engineer for the Agent Factory team to drive end-to-end system design and product integration... ..., and integrating models, balancing latency, cost, and reliability, while implementing governance and data privacy guardrails....
- ...to life! We're building BioRender's AI figure agent, which can create any scientific figure... ...What you'll be doing Lead a small engineering team responsible for the core agent platform... ...to balance quality, cost, speed, and reliability. Build shared AI infrastructure and...Remote work
- ...combines Strategy, Experience & Design, Engineering and Managed Services. We build digital solutions... ....As a Senior Data Engineer, AI & Agents, you will prepare enterprise domain data... ...high data quality, governed access, and reliable agent-based access to trusted data. The...Contract work
$132.6k - $174k per year
...through automation, agentic automation, and AI. We partner with business and... ...Responsibilities As a Senior Software Engineer – AI Agents, you will lead the design and implementation... ...ensuring solutions meet scalability, reliability, and performance requirements. You will...Long term contractPermanent employmentPart timeManual labor- ..., notes, tasks, calendar and they’re not AI-native. Our client’s mission is to build... ....Their product focuses on achieving high reliability for long-running workflows, persistent context... ...meaningful things.Role As a Backend Engineer, AI, you own the inference and...
- Design AI agents and complex software systems at lululemon, using AWS Bedrock, Python, and enterprise... ...four days weekly.As a Senior Software Engineer, you will lead AI agent implementation... ...standards while building scalable, reliable solutions. The Enterprise Automation team...
- Build production AI systems with Python and multi-agent workflows at Guidepoint’s hybrid Toronto office. Shape reliable research technology for global expert networks.As a hands-on Senior AI Engineer, you will move prototypes into production across agent design, evaluation,...Work at office
- Use AI coding agents to deliver reliable applications at Remitly, with measurement, prompt iteration, and product ownership from requirements onward.This Software Engineer II role focuses on directing AI agents to build and ship complete solutions. You will evaluate generated...
- ...Will Do Lead, hire, develop, and scale AI engineering teams and technical leaders supporting... ...engineering quality for LLM-based systems, agents, and integrations, from architecture and... ...operation, cost, latency, and reliability, ensuring teams make sound technical tradeoffs...For contractors
- The Frontier AI Deployed Engineer is a member of Future Electronics' AI & Automation Center of Excellence... ...evaluate emerging AI tools, models, agent capabilities, development methods, and... ...AI solutions for grounding, accuracy, reliability, security, performance, latency,...Work at officeRemote work
- Decagon is building an Agent Deployment Engineering Org to own end-to-end AI agent builds for strategic customers. The role combines hands-on implementation with... ...success, translate requirements into build plans, and ship reliable agent solutions on schedule. #J-18808-Ljbffr
- ...MedMe Health in Toronto is seeking a senior engineer to own high-impact work end-to-end, turning ambiguous goals into shippable milestones... ..., enhance LLM/voice workflows, and raise the bar on reliability and security while collaborating across backend, frontend, and...
- Ema is seeking a Software Engineer, Full Stack in Vancouver to build AI agents in production, writing Go and Python, designing schemas, and presenting solutions... ...and workflows. This role emphasizes ownership, reliability, and platform contributions. You’ll collaborate with...
- Build browser engine capabilities remotely from Canada, using profiling and AI tools to improve reliable programmatic web interaction. Investigate performance with evidence.As a Staff... ...choices with information available to AI agents.Key Responsibilities: Prototype engine...Remote work
- Baseten is seeking a Site Reliability Engineer to define day‑2 operations standards for our ML infrastructure platform. You will build robust systems... ..., and develop observability tooling to keep services reliable at scale while enabling the broader team to operate confidently...
- OverviewWe2; Re looking for a Senior Site Reliability Engineer to help us run one of the most exciting GPU clusters aroundour Toronto datacenter... ...love to hear from you. We may use artificial intelligence (AI) tools to support parts of the hiring process, such as...
Do you want to receive more vacancies?
Subscribe and receive similar vacancies to Ai Agent Reliability Engineer. Be the first to apply!
Related searches
- représentant médical Winnipeg, MB
- représentant interne Winnipeg, MB
- agent nettoyage Winnipeg, MB
- work from home chat agent Winnipeg, MB
- security agent Winnipeg, MB
- représentant représentante en assurances Winnipeg, MB
- agent en santé et sécurité Winnipeg, MB
- représentant senior Winnipeg, MB
- agent des communications Winnipeg, MB
- agent de recherche Winnipeg, MB

