Sign up to access all features of our service.
  • Job search
  • Favorites
  • Create a CV
    New
  • Salaries
  • Subscriptions

Ai Agent Reliability Engineer

TribalScale

Improve AI agent reliability at TribalScale with evaluation pipelines, quality metrics, and disciplined testing for agentic systems. The location remains to be confirmed.This Agentic QA Engineer role focuses on validating AI-driven workflows in real-world environments. You will design evaluation datasets, test scenarios, and benchmarks for dynamic application outputs. You will also collaborate with AI engineers, developers, product teams, and client-facing engineering teams.Key Responsibilities:
Develop pipelines measuring agent accuracy and task completionEstablish quality metrics, datasets, scenarios, and benchmarksEvaluate edge cases, recovery, security, and resilienceIntegrate quality checks into CI/CD pipelinesInvestigate production issues and improve test coverageRequirements:
Strong analytical thinking for complex real-world test casesExperience creating automated tests and testing frameworksStrong API, integration, regression, and end-to-end testing knowledgeExperience debugging applications and technical issuesFamiliarity with modern software development practicesBring quality engineering expertise to AI agent evaluation, production reliability, and measurable software performance at TribalScale.
#J-18808-Ljbffr
Vacancy posted 2 hours ago
Similar jobs that could be interesting for youBased on the Ai Agent Reliability Engineer in Winnipeg, MB vacancy
  • Join Rootly as an AI Engineer specializing in developing reliable AI-driven solutions for incident management. Use your LLM experience to shape the future of business reliability. As a Senior AI Engineer at Rootly, you will lead the development of a transformative AI reliability... 
    Suggested

    Rootly

    Winnipeg, MB
    3 hours ago
  • As a Site Reliability Engineer II on the Serving Platforms team within Infrastructure Engineering, you will design, automate, and manage the core...  ...RabbitMQ) in productionExperience operating, scaling, and monitoring AI/ML or LLM-powered services and workloads in high-concurrency... 
    Suggested
    Worldwide

    OpenTable

    Winnipeg, MB
    2 hours ago
  • Design AI-agent workflows with Claude Code while building reliable engineering harnesses for software-development workflows in Toronto.This Agentic Engineer role focuses on introducing modern agentic engineering practices into a complex existing engineering organization. You... 
    Suggested

    TEEMA Group

    Winnipeg, MB
    2 hours ago
  • Lead AI agent engineering at BioRender remotely, focusing on multi-agent orchestration, model routing, and dependable scientific figure generation...  ...platform with engineering leadership, model evaluation, orchestration, and reliable scientific figure generation. #J-18808-Ljbffr
    Suggested
    Remote work

    BioRender

    Winnipeg, MB
    2 hours ago
  • Shape AI agent platforms in Canada through tool calling, model routing, and production operations. Apply strong engineering judgment to practical systems.This Applied AI Engineer role centers...  ...runtime architecture and orchestration for reliable customer workflows. You will design... 
    Suggested

    Jobgether

    Winnipeg, MB
    2 hours ago
  • Build AI agents for ClickStack using TypeScript, Node.Js, Python, and MCP in a product-focused engineering environment. Improve how developers investigate incidents.As an AI Product...  ...experience, reusable skills, production reliability, and integrations. You will help agents... 

    ClickHouse

    Winnipeg, MB
    2 hours ago
  • $130k - $155k per year

     .... Our vision is to build an ML/AI first network of advertisers and...  ...hiring its first dedicated AI Engineer. Over the past year, AI at Fluent...  ...to production: a service desk agent that closes tickets without a...  ...It is getting trusted data and reliable actions in front of it.You are... 
    Work at office
    Remote work

    Fluent, LLC

    Winnipeg, MB
    2 hours ago
  • Build enterprise AI agents at Decagon in an in-office role requiring technical delivery, integrations...  ...-facing execution.As an Agent Deployment Engineer, you will own strategic customer...  ...role connects customer requirements with reliable AI agent delivery.Key Responsibilities:... 
    Work at office

    FDE Times

    Winnipeg, MB
    2 hours ago
  •  ...commitment to people. Our dependable quality, reliable delivery, and knowledgeable follow-up...  ...and Coffee. Position Summary Join our AI team as an Intermediate AI Engineer, where you'll build and deploy AI applications, agents, and automation solutions that solve real business... 

    Price Industries Limited

    Winnipeg, MB
    5 days ago
  • Build AI agents for renewals and expansion at ShareGate, using low-code orchestration, Python or TypeScript, and SQL remotely across Canada...  ..., marketing operations, customer success operations, or GTM engineering backgroundHands-on n8n, Make, Zapier, or similar experiencePython... 
    Remote work

    Sharegate

    Winnipeg, MB
    2 hours ago
  • Secure AI agents and LLMs in Toronto through cloud security architecture, guardrails, and protected non-human identities. This is a hybrid, full-time opportunity.As an AI security specialist, you will design protections across enterprise finance, ERP, and aerospace and defense... 
    Full time

    Periscope Search Group

    Winnipeg, MB
    2 hours ago
  •  ...MedMe Health in Toronto is seeking a senior engineer to own high-impact work end-to-end, turning ambiguous goals into shippable milestones...  ..., enhance LLM/voice workflows, and raise the bar on reliability and security while collaborating across backend, frontend, and... 

    MedMe Health

    Winnipeg, MB
    2 hours ago
  •  ...Miratech is looking for a skilled Senior AI Engineer with Python, LangGraph and LangChain to...  ...Responsibilities   Develop and enhance AI agents and orchestration capabilities using...  ...technical challenges, improve solution reliability, and drive continuous platform... 
    Full time
    Internship
    Remote work
    Relocation

    Miratech

    Winnipeg, MB
    4 days ago
  • Build production AI agents and order management systems at NewAge Products, working remotely...  ...automation through hands-on software engineering.As an AI Engineer, you will design, test...  ...integration, and evaluation expertise to reliable automation at NewAge Products. #J-18808... 
    Remote work

    NewAge Products Inc.

    Winnipeg, MB
    2 hours ago
  • $132.6k - $174k per year

     ...through automation, agentic automation, and AI. We partner with business and...  ...Core responsibilitiesAs a Senior Software Engineer – AI Agents, you will lead the design and implementation...  ...ensuring solutions meet scalability, reliability, and performance requirements. You will... 
    Long term contract
    Permanent employment
    Part time
    Manual labor
    Local area
    Immediate start

    lululemon

    Winnipeg, MB
    2 hours ago
  •  ...to life! We're building BioRender's AI figure agent, which can create any scientific figure...  ...What you'll be doing Lead a small engineering team responsible for the core agent platform...  ...to balance quality, cost, speed, and reliability. Build shared AI infrastructure and... 
    Remote work

    BioRender

    Winnipeg, MB
    2 hours ago
  • Baseten is seeking a Site Reliability Engineer to define day‑2 operations standards for our ML infrastructure platform. You will build robust systems...  ..., and develop observability tooling to keep services reliable at scale while enabling the broader team to operate confidently... 

    Baseten

    Winnipeg, MB
    2 hours ago
  •  ..., notes, tasks, calendar and they’re not AI-native. Our client’s mission is to build...  ....Their product focuses on achieving high reliability for long-running workflows, persistent context...  ...meaningful things.Role As a Backend Engineer, AI, you own the inference and... 

    Sarsfield Group

    Winnipeg, MB
    2 hours ago
  • The Frontier AI Deployed Engineer is a member of Future Electronics' AI & Automation Center of Excellence...  ...evaluate emerging AI tools, models, agent capabilities, development methods, and...  ...AI solutions for grounding, accuracy, reliability, security, performance, latency,... 
    Work at office
    Remote work

    Future Electronics

    Winnipeg, MB
    2 hours ago
  • Build browser engine capabilities remotely from Canada, using profiling and AI tools to improve reliable programmatic web interaction. Investigate performance with evidence.As a Staff...  ...choices with information available to AI agents.Key Responsibilities: Prototype engine... 
    Remote work

    Mozilla

    Winnipeg, MB
    2 hours ago
  • Use AI coding agents to deliver reliable applications at Remitly, with measurement, prompt iteration, and product ownership from requirements onward.This Software Engineer II role focuses on directing AI agents to build and ship complete solutions. You will evaluate generated... 

    Remitly

    Winnipeg, MB
    2 hours ago
  • Workday, Inc. in Vancouver, Canada is seeking a Senior AI Engineer for the Agent Factory team to drive end-to-end system design and product integration...  ..., and integrating models, balancing latency, cost, and reliability, while implementing governance and data privacy guardrails.... 

    Workday, Inc.

    Winnipeg, MB
    2 hours ago
  • Tenstorrent in Toronto, Canada, is seeking a Staff Reliability Engineer to define the reliability strategy for our next-generation AI computing systems. You’ll drive MTBF modeling, root-cause investigations, and cross-functional collaboration across mechanical, electrical, thermal... 

    Tenstorrent Unlisted/Referral Jobs

    Winnipeg, MB
    2 hours ago
  • Design AI agents and complex software systems at lululemon, using AWS Bedrock, Python, and enterprise...  ...four days weekly.As a Senior Software Engineer, you will lead AI agent implementation...  ...standards while building scalable, reliable solutions. The Enterprise Automation team... 

    Lululemon Athletica Inc.

    Winnipeg, MB
    2 hours ago
  • Build production AI systems with Python and multi-agent workflows at Guidepoint’s hybrid Toronto office. Shape reliable research technology for global expert networks.As a hands-on Senior AI Engineer, you will move prototypes into production across agent design, evaluation,... 
    Work at office

    Guidepoint

    Winnipeg, MB
    2 hours ago
  •  ...Will Do Lead, hire, develop, and scale AI engineering teams and technical leaders supporting...  ...engineering quality for LLM-based systems, agents, and integrations, from architecture and...  ...operation, cost, latency, and reliability, ensuring teams make sound technical tradeoffs... 
    For contractors

    GoodRx, Inc.

    Winnipeg, MB
    2 hours ago
  • Build AI-powered communication experiences with React and TypeScript at EvenUp, working hybrid from Toronto or San Francisco three days weekly.EvenUp seeks Senior and Mid Level Frontend Engineers for its Agents engineering pod within Growth & Agents. You will create user experiences... 

    Evenup

    Winnipeg, MB
    2 hours ago
  • Decagon is building an Agent Deployment Engineering org focused on end-to-end AI agent builds for strategic customers. As an Agent Deployment Engineer you will own the implementation work to launch and scale enterprise-grade agents, partnering with customers and internal teams... 

    FDE Times

    Winnipeg, MB
    2 hours ago
  •  ...Extreme Networks, Inc. is seeking an AI Agent Developer & Marketo Administrator to join our Marketing Operations team. You will focus...  ...integrations across US time zones. This hands-on role blends AI engineering with marketing operations, requiring strong technical skills,... 

    Extreme Networks, Inc

    Winnipeg, MB
    2 hours ago
  • $120k - $220k per year

    Senior Founding Engineer - AI Agents Salary: $120k – $220k CAD 0.25% – 1.5% Job Location: Toronto Experience: 3 years of exp Job Type: Full Time Job LocationToronto Remote Work PolicyIn office - WFH flexibility Visa SponsorshipNot Available RelocationAllowed... 
    Full time
    For contractors
    Internship
    Work at office
    Remote work
    Work from home

    Tyce

    Winnipeg, MB
    2 hours ago

Do you want to receive more vacancies?

Subscribe and receive similar vacancies to Ai Agent Reliability Engineer. Be the first to apply!