Sign up to access all features of our service.
  • Job search
  • Favorites
  • Create a CV
    New
  • Salaries
  • Subscriptions

Llm Application Test Engineer

TribalScale

Test LLM-powered applications at TribalScale using automation, API validation, and failure analysis for complex agent interactions. The location remains to be confirmed.As an Agentic QA Engineer, you will assess applications whose outputs are dynamic, nondeterministic, and dependent on multi-step reasoning. Your work will cover agent tool usage, external systems, integrations, and production behavior. Experience with AI agents or generative AI products is especially relevant.Key Responsibilities:
Validate agent tools, APIs, and external system interactionsDesign end-to-end testing strategies for agentic workflowsReproduce unexpected outputs and tool-use failuresApply automated quality checks to development pipelinesImprove agent behavior through production findingsRequirements:
Professional QA, quality engineering, SDET, or software engineering experienceExperience with automated testing and framework developmentUnderstanding of API and end-to-end testingExperience debugging complex applicationsFamiliarity with Git and CI/CD pipelinesUse software testing, automation, and AI application knowledge to strengthen LLM workflow quality and reliability at TribalScale.
#J-18808-Ljbffr
Vacancy posted 49 minutes ago
Similar jobs that could be interesting for youBased on the Llm Application Test Engineer in Winnipeg, MB vacancy
  • Hygiena in Mississauga, Ontario seeks an Applied AI Engineer to design, build, and deploy GenAI/LLM applications across Hygiena products, including data-driven...  ...align with regulatory standards, and advance MLOps, testing, and deployment practices in a fast-growing environment... 
    Suggested

    Urban Ridge Supplies

    Winnipeg, MB
    50 minutes ago
  • Apply penetration testing and secure code review expertise as an Application Security Engineer at TMX, supporting products across global capital markets.This Application Security...  ..., and risk managementAutomate security tests within CI/CD pipelinesRequirements: Three-plus... 
    Suggested

    tmx

    Winnipeg, MB
    49 minutes ago
  • Test Java and MS .Net web applications with Selenium, SQL, and accessibility knowledge as a hybrid Senior Web Application Test Engineer. Onsite work is required three days weekly.This senior testing specialist develops maintainable test plans, test cases, automation scripts... 
    Suggested

    Rubicon Path

    Winnipeg, MB
    51 minutes ago
  •  ...passion to drive life-changing impact to ZS.LLM Engineer We are seeking a highly skilled LLM...  ...requirements analysis, design, coding, testing, and deployment.Utilize AWS services/...  ...Search, etc.Strong understanding of scalable application design principles and experience with... 
    Suggested
    Work at office
    Local area
    Work from home
    Worldwide
    Flexible hours

    ZS

    Winnipeg, MB
    53 minutes ago
  • $114.4k - $164.4k per year

    Company:Qualcomm Canada ULC Job Area:Engineering Group, Engineering Group Machine Learning...  ...development and optimization with application to a specific problem domain in ML (e.G....  ...expects its employees to abide by all applicable policies and procedures, including but not... 
    Suggested
    Immediate start
    Work from home

    Urban Ridge Supplies

    Winnipeg, MB
    48 minutes ago
  •  ...4 years’ experience with creating other project documentation related to testing such as writing test strategies, test scenarios and test cases 4 years’ experience with testing various applications, monitoring testing activities and providing guidance/support to the test... 
    Contract work

    TecTammina

    Winnipeg, MB
    51 minutes ago
  • Build LLM-powered and agentic features for Quartermaster using Python...  .... This is hands-on software engineering.As an AI Engineer, you will...  ...capabilities across a production SaaS application. You will work with LLM APIs,...  ...prompts, agents, tests, deployments, and monitoringEvaluate... 

    hireVouch

    Winnipeg, MB
    47 minutes ago
  • $195k - $390k per year

     ...AI!What you'll be doing:Work closely with senior engineers to deliver, optimize, maintain, debug, unit test and document next-generation dialogue and security...  ...to craft, develop, test, and maintain integrated applications and featuresActively contribute to the adoption of... 
    Flexible hours

    NVIDIA Corporation

    Winnipeg, MB
    44 minutes ago
  • Python and LLM engineering define this remote Senior AI/LLM Engineer position with Kake, building production-ready applications and systems at scale.Kake seeks senior engineers who design, build...  ..., and AI output evaluation testing experienceApply senior Python engineering... 
    Remote work

    Kake

    Winnipeg, MB
    49 minutes ago
  • Use LLM tools, JavaScript, and solution design to extend Orgvue’s organizational planning...  ..., and internal teams.As Senior Solutions Engineer, you will help define, develop, and deliver...  ...LLM experience writing instructions and testing performanceStrong technical consulting and... 
    Remote work

    Orgvue

    Winnipeg, MB
    52 minutes ago
  • Oracle Fusion Applications Test Lead (FSCM, EPM, Procurement, OTBI) Oracle Fusion Applications Test Lead (FSCM,EPM, Procurement, OTBI) Experience Required: 6–8 Years Role Descriptions: Specialized Fusion Testing Testend-to-end business processes within and across... 

    Astra-North Infoteck Inc.

    Winnipeg, MB
    55 minutes ago
  • About the RoleWe are seeking a versatile and experienced engineer to join our Inference Core Model Bringup team. This team is responsible...  ...levels of performance, efficiency, and scalability for AI applications. Responsibilities Contribute to the end-to‑end bring up of... 

    Cerebras

    Winnipeg, MB
    48 minutes ago
  • Drive model integration projects as a Senior Engineer focused on AI LLMs. Engage in end-to-end processes from architecture translation...  ...contributions will not only enhance the systems but also transform AI applications as a whole. Key Responsibilities: • Spearhead bringing ML... 

    Cerebras

    Winnipeg, MB
    55 minutes ago
  • $218.89k - $245.95k per year

     ...conversations so they can debug them. We're bringing LLM and agent observability together, and we...  ...in a small, cross-functional group of engineers with a dedicated product manager and...  ...time.Additionally, in compliance with applicable law, all persons hired will be required to... 
    Internship
    Work at office
    Local area
    Work from home
    Home office
    Visa sponsorship
    Shift work

    Candidate

    Winnipeg, MB
    49 minutes ago
  • Design RAG systems and LLM features for Wiser Solutions remotely, using embeddings...  ...problems.This Senior Machine Learning Engineer builds AI applications that reach production across product...  ...Your work includes evaluation, rollout testing, and live-system improvement.Key... 
    Remote work

    Wiser Solutions

    Winnipeg, MB
    53 minutes ago
  • Build reliable LLM platforms in Montreal with GitHub Copilot, Python, and CI/CD...  ...role centers on production AI engineering.This Senior AI Application Developer position delivers AI and LLM...  ...position also addresses observability, testing, security, responsible AI, and operational... 

    Fujitsu

    Winnipeg, MB
    50 minutes ago
  • Develop LLM workflows and data pipelines for AXIS Capital’s enterprise...  ...teams.This Senior AI Platform Engineer builds reusable capabilities...  ...: Strong maintainable and tested Python engineering...  ...Databricks workloads and pipelinesLLM application or agent framework experience... 

    AXIS Capital

    Winnipeg, MB
    51 minutes ago
  • Python and LLM application engineering define this backend role at Supra ITS. Build production agent runtimes for cloud and on-premise deployments.You...  ...services, LangChain or LangGraph, SQL, Postgres, and Test Driven Development. You will ship through stories, architecture... 

    Supra ITS

    Winnipeg, MB
    51 minutes ago
  •  ...you’re invited to be a part of that. Applications Engineering Specialist – Antec Controls We’re...  ...records in CRM System • Perform product testing to support the implementation of new...  ...appreciate the interest shown by all applicants, however only those being considered... 
    For contractors
    Immediate start

    Price Industries Limited

    Winnipeg, MB
    10 days ago
  • Build advanced language systems as a Senior AI Engineer in Kimberley, applying NLP and LLM expertise within a hybrid work model. Develop scalable solutions that understand, generate, and process human language at scale.This senior position focuses on developing and deploying... 

    Placements24

    Winnipeg, MB
    44 minutes ago
  • Apply LLM, RAG, and software safety expertise at CS Group Canada in a hybrid, full-time contract focused on SW-FMEA automation.As a Software Safety Engineer, you will support the ZHAP improvement initiative for two months. The work combines artificial intelligence, C/C++ analysis... 
    Full time
    Contract work

    Sopra Steria

    Winnipeg, MB
    52 minutes ago
  • Engineer production LLM systems for ClickStack with agent design, evaluation, and SQL expertise. Work on observability tools used to understand real telemetry.ClickHouse is hiring an AI Product Engineer to develop agents on ClickStack, its open-source observability platform... 

    ClickHouse

    Winnipeg, MB
    44 minutes ago
  •  ...through systems thinking, exploratory testing, and production-code debugging....  ...Content Pillar team. You will work with Engineering, Product, and Customer teams on LLM-backed systems, data pipelines, and...  ...includes user-centered discovery, test strategy, observability, and... 
    Work at office
    Remote work

    Blue J

    Winnipeg, MB
    48 minutes ago
  • $100k - $130k per year

    Role Overview DarkVision is seeking a highly organized Applications Engineer to manage the testing and validation of our award-winning industrial imaging...  ...effort.Translate the desired outcome into an actionable test plan – ideate samples and strategize to reduce time to data... 
    Work at office

    DarkVision

    Winnipeg, MB
    52 minutes ago
  • $85k - $105k per year

    If you are unable to complete this application due to a disability, contact this employer to ask...  .... Applications EngineerFull Time Engineering Ontario, Burlington, ON, CA 3 days ago...  ...: Circuit Check is a global leader in test systems and engineering solutions for complex... 
    Hourly pay
    Long term contract
    Temporary work
    Work at office

    Circuit Check Inc

    Winnipeg, MB
    55 minutes ago
  • Lead LLM systems at Axya, working hands-on with Python, retrieval, agent workflows, and...  ...evaluation in a 100% remote environment.As Lead AI Engineer, you will own Axya’s AI architecture and...  ...use workflowsBuild evaluation frameworks, test sets, and monitoringDeliver observable,... 
    Remote work

    Axya

    Winnipeg, MB
    52 minutes ago
  • Senior Machine Learning Engineer – LLM’sTech scale up developing democratized AI Technology, backed by one of the worlds leading AI investors...  ...Python.Familiar with object-oriented design methodology and application development in Python.Experience and good understanding of Transformer... 

    Wyatt Partners

    Winnipeg, MB
    50 minutes ago
  •  ...You Will Conduct manual penetration tests against each of our systems, and build the...  ...AI. Bring us new ideas for testing LLM applications, agents, and agent-generated code, and build...  ...reviewed before promotion.Explain risk to engineers in enough detail that they can... 
    Long term contract
    Work at office
    Remote work
    Work from home
    Flexible hours

    Forward Financing LLC

    Winnipeg, MB
    46 minutes ago
  • The Senior Application Security Engineer is responsible for helping secure the Paymentus SaaS platform...  ...assessments, application security testing, vulnerability validation, and remediation...  ...supply chain risk, and emerging AI/LLM application security risks. Supervisory... 
    Local area

    Paymentus

    Winnipeg, MB
    50 minutes ago
  • The Senior Application Security Engineer is responsible for helping secure the Paymentus SaaS platform...  ...assessments, application security testing, vulnerability validation, and remediation...  ...supply chain risk, and emerging AI/LLM application security risks.Essential Functions... 
    Full time
    Work at office
    Local area
    Monday to friday
    Afternoon shift

    Paymentus Corporation

    Winnipeg, MB
    53 minutes ago

Do you want to receive more vacancies?

Subscribe and receive similar vacancies to Llm Application Test Engineer. Be the first to apply!