Sign up to access all features of our service.
  • Job search
  • Favorites
  • Create a CV
    New
  • Salaries
  • Subscriptions

Ai Agent Reliability Engineer

TribalScale

Improve AI agent reliability at TribalScale with evaluation pipelines, quality metrics, and disciplined testing for agentic systems. The location remains to be confirmed.This Agentic QA Engineer role focuses on validating AI-driven workflows in real-world environments. You will design evaluation datasets, test scenarios, and benchmarks for dynamic application outputs. You will also collaborate with AI engineers, developers, product teams, and client-facing engineering teams.Key Responsibilities:
Develop pipelines measuring agent accuracy and task completionEstablish quality metrics, datasets, scenarios, and benchmarksEvaluate edge cases, recovery, security, and resilienceIntegrate quality checks into CI/CD pipelinesInvestigate production issues and improve test coverageRequirements:
Strong analytical thinking for complex real-world test casesExperience creating automated tests and testing frameworksStrong API, integration, regression, and end-to-end testing knowledgeExperience debugging applications and technical issuesFamiliarity with modern software development practicesBring quality engineering expertise to AI agent evaluation, production reliability, and measurable software performance at TribalScale.
#J-18808-Ljbffr
Vacancy posted 2 hours ago
Similar jobs that could be interesting for youBased on the Ai Agent Reliability Engineer in Winnipeg, MB vacancy
  • Join Rootly as an AI Engineer specializing in developing reliable AI-driven solutions for incident management. Use your LLM experience to shape the future of business reliability. As a Senior AI Engineer at Rootly, you will lead the development of a transformative AI reliability... 
    Suggested

    Rootly

    Winnipeg, MB
    2 hours ago
  • $102k - $152k per year

     ...Equinix's GenAI products and platforms stable and reliable, and makes them more resilient over time....  ...a lasting fix. Works in a team with senior engineers and with product, platform and security partners to keep AI products, agents, connectors and self-hosted models robust,... 
    Suggested
    Full time
    Manual labor
    Work at office

    Equinix

    Winnipeg, MB
    2 hours ago
  • Build enterprise AI agents at Decagon in an in-office role requiring technical delivery, integrations...  ...-facing execution.As an Agent Deployment Engineer, you will own strategic customer...  ...role connects customer requirements with reliable AI agent delivery.Key Responsibilities:... 
    Suggested
    Work at office

    FDE Times

    Winnipeg, MB
    2 hours ago
  • $130k - $155k per year

     .... Our vision is to build an ML/AI first network of advertisers and...  ...hiring its first dedicated AI Engineer. Over the past year, AI at Fluent...  ...to production: a service desk agent that closes tickets without a...  ...It is getting trusted data and reliable actions in front of it.You are... 
    Suggested
    Work at office
    Remote work

    Fluent, LLC

    Winnipeg, MB
    2 hours ago
  • Lead AI agent engineering at BioRender remotely, focusing on multi-agent orchestration, model routing, and dependable scientific figure generation...  ...platform with engineering leadership, model evaluation, orchestration, and reliable scientific figure generation. #J-18808-Ljbffr
    Suggested
    Remote work

    BioRender

    Winnipeg, MB
    2 hours ago
  • Build AI agents for ClickStack using TypeScript, Node.Js, Python, and MCP in a product-focused engineering environment. Improve how developers investigate incidents.As an AI Product...  ...experience, reusable skills, production reliability, and integrations. You will help agents... 

    ClickHouse

    Winnipeg, MB
    2 hours ago
  • Shape AI agent platforms in Canada through tool calling, model routing, and production operations. Apply strong engineering judgment to practical systems.This Applied AI Engineer role centers...  ...runtime architecture and orchestration for reliable customer workflows. You will design... 

    Jobgether

    Winnipeg, MB
    2 hours ago
  • As a Site Reliability Engineer II on the Serving Platforms team within Infrastructure Engineering, you will design, automate, and manage the core...  ...RabbitMQ) in productionExperience operating, scaling, and monitoring AI/ML or LLM-powered services and workloads in high-concurrency... 
    Worldwide

    OpenTable

    Winnipeg, MB
    2 hours ago
  • Lead AI-powered reliability engineering at Questrade, using Python and machine learning to strengthen high-frequency trading platforms in a hybrid environment...  ...work combines Google Cloud Platform, Google Kubernetes Engine, Terraform, Helm, observability, and intelligent... 

    Questrade Financial Group

    Winnipeg, MB
    2 hours ago
  • Design AI-agent workflows with Claude Code while building reliable engineering harnesses for software-development workflows in Toronto.This Agentic Engineer role focuses on introducing modern agentic engineering practices into a complex existing engineering organization. You... 

    TEEMA Group

    Winnipeg, MB
    2 hours ago
  •  ...commitment to people. Our dependable quality, reliable delivery, and knowledgeable follow-up...  ...and Coffee. Position Summary Join our AI team as an Intermediate AI Engineer, where you'll build and deploy AI applications, agents, and automation solutions that solve real business... 

    Price Industries Limited

    Winnipeg, MB
    9 days ago
  • Build production AI agents for ClickStack with TypeScript, Python, MCP, and a developer experience focus at ClickHouse. Work remotely across ClickHouse’s distributed company.As an AI Product Engineer, you will create agentic capabilities on ClickStack, ClickHouse’s open-source... 
    Remote work

    ClickHouse

    Winnipeg, MB
    2 hours ago
  • Build AI agents for renewals and expansion at ShareGate, using low-code orchestration, Python or TypeScript, and SQL remotely across Canada...  ..., marketing operations, customer success operations, or GTM engineering backgroundHands-on n8n, Make, Zapier, or similar experiencePython... 
    Remote work

    Sharegate

    Winnipeg, MB
    2 hours ago
  • Lead AI figure agent engineering at BioRender, focusing on multi-agent orchestration and dependable scientific figure generation. This remote-first...  ..., planning, tool use, self-correction, and production reliability. The work combines frontier LLM and VLM evaluation, model... 
    Remote work

    BioRender

    Winnipeg, MB
    2 hours ago
  • Workday, Inc. in Vancouver, Canada is seeking a Senior AI Engineer for the Agent Factory team to drive end-to-end system design and product integration...  ..., and integrating models, balancing latency, cost, and reliability, while implementing governance and data privacy guardrails.... 

    Workday, Inc.

    Winnipeg, MB
    2 hours ago
  •  ...to life! We're building BioRender's AI figure agent, which can create any scientific figure...  ...What you'll be doing Lead a small engineering team responsible for the core agent platform...  ...to balance quality, cost, speed, and reliability. Build shared AI infrastructure and... 
    Remote work

    BioRender

    Winnipeg, MB
    2 hours ago
  •  ...combines Strategy, Experience & Design, Engineering and Managed Services. We build digital solutions...  ....As a Senior Data Engineer, AI & Agents, you will prepare enterprise domain data...  ...high data quality, governed access, and reliable agent-based access to trusted data. The... 
    Contract work

    Appnovation Technologies

    Winnipeg, MB
    2 hours ago
  • $132.6k - $174k per year

     ...through automation, agentic automation, and AI. We partner with business and...  ...Responsibilities As a Senior Software Engineer – AI Agents, you will lead the design and implementation...  ...ensuring solutions meet scalability, reliability, and performance requirements. You will... 
    Long term contract
    Permanent employment
    Part time
    Manual labor

    Lululemon Athletica Inc.

    Winnipeg, MB
    2 hours ago
  •  ..., notes, tasks, calendar and they’re not AI-native. Our client’s mission is to build...  ....Their product focuses on achieving high reliability for long-running workflows, persistent context...  ...meaningful things.Role As a Backend Engineer, AI, you own the inference and... 

    Sarsfield Group

    Winnipeg, MB
    2 hours ago
  • Design AI agents and complex software systems at lululemon, using AWS Bedrock, Python, and enterprise...  ...four days weekly.As a Senior Software Engineer, you will lead AI agent implementation...  ...standards while building scalable, reliable solutions. The Enterprise Automation team... 

    Lululemon Athletica Inc.

    Winnipeg, MB
    2 hours ago
  • Build production AI systems with Python and multi-agent workflows at Guidepoint’s hybrid Toronto office. Shape reliable research technology for global expert networks.As a hands-on Senior AI Engineer, you will move prototypes into production across agent design, evaluation,... 
    Work at office

    Guidepoint

    Winnipeg, MB
    2 hours ago
  • Use AI coding agents to deliver reliable applications at Remitly, with measurement, prompt iteration, and product ownership from requirements onward.This Software Engineer II role focuses on directing AI agents to build and ship complete solutions. You will evaluate generated... 

    Remitly

    Winnipeg, MB
    2 hours ago
  •  ...Will Do Lead, hire, develop, and scale AI engineering teams and technical leaders supporting...  ...engineering quality for LLM-based systems, agents, and integrations, from architecture and...  ...operation, cost, latency, and reliability, ensuring teams make sound technical tradeoffs... 
    For contractors

    GoodRx, Inc.

    Winnipeg, MB
    2 hours ago
  • The Frontier AI Deployed Engineer is a member of Future Electronics' AI & Automation Center of Excellence...  ...evaluate emerging AI tools, models, agent capabilities, development methods, and...  ...AI solutions for grounding, accuracy, reliability, security, performance, latency,... 
    Work at office
    Remote work

    Future Electronics

    Winnipeg, MB
    2 hours ago
  • Decagon is building an Agent Deployment Engineering Org to own end-to-end AI agent builds for strategic customers. The role combines hands-on implementation with...  ...success, translate requirements into build plans, and ship reliable agent solutions on schedule. #J-18808-Ljbffr

    Urban Ridge Supplies

    Winnipeg, MB
    2 hours ago
  •  ...MedMe Health in Toronto is seeking a senior engineer to own high-impact work end-to-end, turning ambiguous goals into shippable milestones...  ..., enhance LLM/voice workflows, and raise the bar on reliability and security while collaborating across backend, frontend, and... 

    MedMe Health

    Winnipeg, MB
    2 hours ago
  • Ema is seeking a Software Engineer, Full Stack in Vancouver to build AI agents in production, writing Go and Python, designing schemas, and presenting solutions...  ...and workflows. This role emphasizes ownership, reliability, and platform contributions. You’ll collaborate with... 

    MetAntz

    Winnipeg, MB
    2 hours ago
  • Build browser engine capabilities remotely from Canada, using profiling and AI tools to improve reliable programmatic web interaction. Investigate performance with evidence.As a Staff...  ...choices with information available to AI agents.Key Responsibilities: Prototype engine... 
    Remote work

    Mozilla

    Winnipeg, MB
    2 hours ago
  • Baseten is seeking a Site Reliability Engineer to define day‑2 operations standards for our ML infrastructure platform. You will build robust systems...  ..., and develop observability tooling to keep services reliable at scale while enabling the broader team to operate confidently... 

    Baseten

    Winnipeg, MB
    2 hours ago
  • OverviewWe2; Re looking for a Senior Site Reliability Engineer to help us run one of the most exciting GPU clusters aroundour Toronto datacenter...  ...love to hear from you. We may use artificial intelligence (AI) tools to support parts of the hiring process, such as... 

    Boson AI

    Winnipeg, MB
    2 hours ago

Do you want to receive more vacancies?

Subscribe and receive similar vacancies to Ai Agent Reliability Engineer. Be the first to apply!