Agentic Qa Engineer
TribalScale
TribalScale is a digital innovation firm that helps organizations adapt and thrive in a rapidly changing world. We partner with clients to build transformative digital products, modernize technology platforms, and apply emerging technologies—including artificial intelligence—to meaningful business challenges.Our teams work closely with clients from strategy through execution. We value curiosity, ownership, practical innovation, and people who are comfortable turning ambiguity into action.
Our Digital Expertise
Transformation Experts:
We transform traditional business models into agile, AI-powered ecosystems.
Strategic Visionaries:
We navigate the uncharted waters of technological evolution.
Product Virtuosos:
We orchestrate the creation of world-class digital solutions.
Code Optimizers:
We leverage AI to refine and perfect digital systems.
Our Technological Toolkit
Voice-Activated Platforms:
(Amazon Alexa, Google Home)
Connected Ecosystems:
(Cars and Homes)
The TribalScale ProfessionalYou thrive in challenging environments where innovation is key. Your passion for technology is matched only by your drive to redefine its limits. You seek more than a job;
you seek a calling—a chance to make a lasting impact on the digital landscape.
Agentic QA EngineerJob title: Agentic QA EngineerCompany: TribalScaleLocation: [To be confirmed]Employment type: [To be confirmed]
About TribalScaleTribalScale is an innovation company focused on helping organizations build and scale digital products and AI-powered solutions. The team works at the intersection of emerging technologies, product development, and engineering to bring innovative solutions into production.
About the RoleTribalScale is looking for an Agentic QA Engineer who combines strong software testing and quality engineering expertise with hands-on experience in AI-powered applications and agentic systems.
This role is similar to a Forward Deployed Engineer (FDE) or Agentic Engineer, with a particular focus on software quality, test automation, and validating AI-driven workflows. You will help ensure that AI agents and LLM-powered applications perform reliably in real-world environments by designing testing strategies, building automated evaluation frameworks, and identifying edge cases and failure modes.
The ideal candidate is a hands-on engineer who can write code, debug complex systems, and apply software testing principles to applications where outputs may be dynamic, nondeterministic, and dependent on multi-step agent interactions.
Key Responsibilities
Design and implement testing strategies for AI agents, LLM-powered applications, and agentic workflows.
Build automated test frameworks for functional, integration, regression, and end-to-end testing.
Develop evaluation pipelines to measure agent accuracy, reliability, task completion, and consistency.
Test agent tool usage, API integrations, multi-step reasoning workflows, and interactions with external systems.
Identify, reproduce, and troubleshoot failures such as hallucinations, incorrect tool calls, unexpected outputs, and workflow breakdowns.
Establish quality metrics, evaluation datasets, test scenarios, and benchmarks for AI-driven applications.
Integrate automated testing and quality checks into CI/CD pipelines.
Collaborate with AI engineers, software developers, product teams, and client-facing engineering teams to improve system reliability.
Investigate production issues and use findings to improve agent behavior, test coverage, and application performance.
Evaluate edge cases, failure recovery, security considerations, and resilience in AI-powered systems.
Required Qualifications
Professional experience in Software QA, Quality Engineering, SDET, Test Automation, or Software Engineering with a strong testing focus.
Experience building automated tests and testing frameworks.
Strong understanding of API, integration, regression, and end-to-end testing.
Experience debugging applications and investigating complex technical issues.
Familiarity with Git, CI/CD pipelines, and modern software development practices.
Strong analytical thinking and the ability to design test cases for complex, real-world scenarios.
Preferred Qualifications
Experience testing LLM-based applications, AI agents, or generative AI products.
Familiarity with agentic frameworks such as LangGraph, LangChain, CrewAI, or similar technologies.
Experience with LLM evaluation, prompt testing, model benchmarking, or AI observability.
Familiarity with tools such as Pytest, Playwright, Cypress, or Selenium.
Understanding of hallucinations, nondeterministic outputs, prompt injection, and agent tool-use failures.
Experience with cloud platforms, APIs, distributed systems, and production deployments.
Previous experience in Agentic Engineering, Forward Deployed Engineering, Applied AI Engineering, or AI product development.
Ideal CandidateThe ideal candidate combines the engineering mindset of an FDE or Agentic Engineer with the quality-focused approach of an experienced QA Automation Engineer or SDET.
Candidates with strong programming and automation skills who have tested AI-powered applications are especially relevant. We are also open to Agentic Engineers or FDEs who have meaningful experience building testing frameworks, validating complex workflows, and improving software reliability.
Core focus: AI agent reliability, software testing, test automation, and production-ready AI systems.Why Choose TribalScaleAt TribalScale, collaboration is at the heart of our innovative approach. We don't simply work for clients;
we form partnerships, building transformative solutions that redefine what's possible. Our team, once at the forefront of the mobile revolution, now leads the charge in AI-driven innovation.
We empower each team member to be a catalyst for change, adhering to our principles of meritocracy, empowerment, and transparency. Diversity isn't just celebrated—it's fundamental to our strength. Our mission to "Right the Future" is a call to action to make a significant impact in the world of innovation.
Diversity: Our Collective StrengthTribalScale proudly stands as a beacon of equal opportunity in employment and workplace culture. We are steadfastly committed to equal employment opportunity, transcending the boundaries of race, color, ancestry, religion, sex, national origin, sexual orientation, age, citizenship, marital status, able-bodiedness, or gender identity.
We not only welcome but actively encourage applications from individuals of all abilities, ages, citizenships, marital statuses, and gender identities. Your unique perspective is the key to our collective innovation.
Should you require any accommodations, we implore you to inform us. Our commitment to inclusivity knows no bounds, and we stand ready to adapt our processes to ensure your comfort and success.
#J-18808-Ljbffr
Our Digital Expertise
Transformation Experts:
We transform traditional business models into agile, AI-powered ecosystems.
Strategic Visionaries:
We navigate the uncharted waters of technological evolution.
Product Virtuosos:
We orchestrate the creation of world-class digital solutions.
Code Optimizers:
We leverage AI to refine and perfect digital systems.
Our Technological Toolkit
Voice-Activated Platforms:
(Amazon Alexa, Google Home)
Connected Ecosystems:
(Cars and Homes)
The TribalScale ProfessionalYou thrive in challenging environments where innovation is key. Your passion for technology is matched only by your drive to redefine its limits. You seek more than a job;
you seek a calling—a chance to make a lasting impact on the digital landscape.
Agentic QA EngineerJob title: Agentic QA EngineerCompany: TribalScaleLocation: [To be confirmed]Employment type: [To be confirmed]
About TribalScaleTribalScale is an innovation company focused on helping organizations build and scale digital products and AI-powered solutions. The team works at the intersection of emerging technologies, product development, and engineering to bring innovative solutions into production.
About the RoleTribalScale is looking for an Agentic QA Engineer who combines strong software testing and quality engineering expertise with hands-on experience in AI-powered applications and agentic systems.
This role is similar to a Forward Deployed Engineer (FDE) or Agentic Engineer, with a particular focus on software quality, test automation, and validating AI-driven workflows. You will help ensure that AI agents and LLM-powered applications perform reliably in real-world environments by designing testing strategies, building automated evaluation frameworks, and identifying edge cases and failure modes.
The ideal candidate is a hands-on engineer who can write code, debug complex systems, and apply software testing principles to applications where outputs may be dynamic, nondeterministic, and dependent on multi-step agent interactions.
Key Responsibilities
Design and implement testing strategies for AI agents, LLM-powered applications, and agentic workflows.
Build automated test frameworks for functional, integration, regression, and end-to-end testing.
Develop evaluation pipelines to measure agent accuracy, reliability, task completion, and consistency.
Test agent tool usage, API integrations, multi-step reasoning workflows, and interactions with external systems.
Identify, reproduce, and troubleshoot failures such as hallucinations, incorrect tool calls, unexpected outputs, and workflow breakdowns.
Establish quality metrics, evaluation datasets, test scenarios, and benchmarks for AI-driven applications.
Integrate automated testing and quality checks into CI/CD pipelines.
Collaborate with AI engineers, software developers, product teams, and client-facing engineering teams to improve system reliability.
Investigate production issues and use findings to improve agent behavior, test coverage, and application performance.
Evaluate edge cases, failure recovery, security considerations, and resilience in AI-powered systems.
Required Qualifications
Professional experience in Software QA, Quality Engineering, SDET, Test Automation, or Software Engineering with a strong testing focus.
Experience building automated tests and testing frameworks.
Strong understanding of API, integration, regression, and end-to-end testing.
Experience debugging applications and investigating complex technical issues.
Familiarity with Git, CI/CD pipelines, and modern software development practices.
Strong analytical thinking and the ability to design test cases for complex, real-world scenarios.
Preferred Qualifications
Experience testing LLM-based applications, AI agents, or generative AI products.
Familiarity with agentic frameworks such as LangGraph, LangChain, CrewAI, or similar technologies.
Experience with LLM evaluation, prompt testing, model benchmarking, or AI observability.
Familiarity with tools such as Pytest, Playwright, Cypress, or Selenium.
Understanding of hallucinations, nondeterministic outputs, prompt injection, and agent tool-use failures.
Experience with cloud platforms, APIs, distributed systems, and production deployments.
Previous experience in Agentic Engineering, Forward Deployed Engineering, Applied AI Engineering, or AI product development.
Ideal CandidateThe ideal candidate combines the engineering mindset of an FDE or Agentic Engineer with the quality-focused approach of an experienced QA Automation Engineer or SDET.
Candidates with strong programming and automation skills who have tested AI-powered applications are especially relevant. We are also open to Agentic Engineers or FDEs who have meaningful experience building testing frameworks, validating complex workflows, and improving software reliability.
Core focus: AI agent reliability, software testing, test automation, and production-ready AI systems.Why Choose TribalScaleAt TribalScale, collaboration is at the heart of our innovative approach. We don't simply work for clients;
we form partnerships, building transformative solutions that redefine what's possible. Our team, once at the forefront of the mobile revolution, now leads the charge in AI-driven innovation.
We empower each team member to be a catalyst for change, adhering to our principles of meritocracy, empowerment, and transparency. Diversity isn't just celebrated—it's fundamental to our strength. Our mission to "Right the Future" is a call to action to make a significant impact in the world of innovation.
Diversity: Our Collective StrengthTribalScale proudly stands as a beacon of equal opportunity in employment and workplace culture. We are steadfastly committed to equal employment opportunity, transcending the boundaries of race, color, ancestry, religion, sex, national origin, sexual orientation, age, citizenship, marital status, able-bodiedness, or gender identity.
We not only welcome but actively encourage applications from individuals of all abilities, ages, citizenships, marital statuses, and gender identities. Your unique perspective is the key to our collective innovation.
Should you require any accommodations, we implore you to inform us. Our commitment to inclusivity knows no bounds, and we stand ready to adapt our processes to ensure your comfort and success.
#J-18808-Ljbffr
Vacancy posted 2 hours ago
Similar jobs that could be interesting for youBased on the Agentic Qa Engineer in Winnipeg, MB vacancy
- We are looking for an Automation Engineer - a developer-first, agentic thinker who will build the intelligent systems that keep Sophis, strategic front-to-back trading platform, running at the highest quality bar. You will operate as an AI agent coordinator, software developer...SuggestedInternship
- .... Our capabilities span generative and agentic AI, cloud migration and modernization, cloud... ...standard verification Partner with engineers during development to shift testing left:... ...assets (automated suites, test data sets, QA checklists) back to the CXE practice Mentor...SuggestedContract workWork at officeRemote workWorldwideFlexible hoursShift work
- ...day inspired to build a more efficient and enjoyable world. Role overviewAgentic engineering changes how software is built and how organizations operate. We're designing and building agentic engineering systems that can reason, use tools, interact with enterprise...SuggestedFlexible hours
- ...from top-tier VCs, and are live with customers like Scale AI, Rippling, and Zendesk. Think of Maxima to accountants as Claude Code to engineers.Responsibilities Increasing end-to-end test coverage (Playwright + use AI to generate tests)Maintaining existing testsManual...Suggested
- ...secretarial, governance, tax, and compliance. About The RoleWe are seeking a detail-oriented and technically skilled Quality Assurance (QA) Engineer to join our growing team. Reporting into the Director of Development, you will play a vital role in ensuring our software products...SuggestedLong term contractFull timeWork at officeRemote workWork from homeFlexible hours
- ...and automation to deliver innovative solutions in a fast-paced environment.About This Opportunity: We’re seeking an experienced QA Engineer to join a dynamic team focused on test automation and AI-driven quality assurance. You’ll collaborate with product managers and development...
$90k - $120k per year
...Ventures, Intel Capital, and Y Combinator. What You'll Do Own the QA strategy across manual, automated, regression, and integration... ...and maintain test frameworks that scale with our product and engineering growth. Triage and validate issues in collaboration with Engineering...Work at office- ...efficient than ever before.Continuous protection. AI-powered Nonstop SecOps. That’s Ontinue.Role Overview We’re looking for a QA Automation Engineer to help us scale quality across a fast-moving engineering team. This role is about building the automation, processes, and...
$175.6k - $283.1k per year
Camunda is the enterprise platform for agentic orchestration, enabling organizations to coordinate AI agents, people, and systems across... ...reading. About The RoleCamunda is hiring a Senior Manager, QA Engineering to lead the end-to-end QA strategy for Camunda 8 - elevating...Work at officeLocal areaRemote workWork from homeWorldwideHome officeFlexible hours- Job DescriptionWe are seeking an experienced Senior QA Engineer (AI Native) to join Semios' Engineering team. This position will take ownership... ...data. Experience supporting teams adopting AI-native or agentic software development practices. About AffinityAffinity Group...Long term contractContract workInternshipImmediate startRemote work
- Nothing can limit yourdreams when you are atZoft SENIOR QA ENGINEER Work Location : Canada We are hiring a Senior QA Manager for an onsite assignment at our client's premises in Canada, owning QA leadership across multiple product streams. You will own end-to-end test...Contract work
- Location: Remote / Hybrid [Insert City if applicable] Type: Full-time About the RoleGrant Advance is looking for a Senior QA Engineer to act as the primary "Gatekeeper" of our platform. We are looking for more than just a tester; we need a partner for our...Long term contractFull timeImmediate startRemote workShift work
- ...TribalScale is looking for a Forward Deployed Engineer to work at the intersection of software... ...of automated delivery cycles Drive agentic engineering practices across our clients,... ...Experience with agent frameworks, workflow engines, coding agents, or custom model‑...Immediate start
- ...using Python, LLM evaluation, and observability for production AI systems.This newly created full-time role focuses on Agentic AI Evaluation and Tuning Engineering. You will test multi-agent architectures, audit decision-making logic, and optimize tool use. The work connects...Full time
- Advance agentic reinforcement learning research from Canada with PyTorch and multimodal AI expertise in a fully remote setting. Build intelligent systems for complex challenges.This AI Research Engineer role focuses on next-generation intelligent systems and adaptive behavior...Remote work
- ...intersection of consulting, data science, AI technologies, data engineering, and digital transformation. We do not just advise — we build,... ...AI adoption. The RoleArtefact is looking for a Senior AI & Agentic Engineer: a full-stack engineer who takes AI features from idea...
- ...position is listed on behalf of a partner company, who manages all applications and next steps. Our partner is looking for an AI Engineer (Agentic Systems) based in Canada. Join a high-performing team of engineers and data scientists working at the forefront of artificial...
- ...solving skills are essential for this role. Degree in Computer Science or Engineering. Strong Python skills and experience building production-grade AI/ML systems. Experience architecting complex agentic systems, including orchestration, tool use, memory, and state management...
- Build LLM-powered and agentic features for Quartermaster using Python, TypeScript, prompting, and API-based model integration. This is hands-on software engineering.As an AI Engineer, you will ship intelligent capabilities across a production SaaS application. You will work...
- Build agentic workflows and personalized AI features as an AI Engineer supporting Superhuman Mail from a remote-flexible Canada arrangement. Apply Python or TypeScript to production AI.This AI Engineer role focuses on backend infrastructure and applied AI for advanced email...Remote workFlexible hours
$98k - $138k per year
...teams deliver innovative technology solutions to help build a stronger financial future for the healthcare community. As a Lead Agentic AI Engineer, you will work directly with engineers, architects, security, governance, and business partners to find high - value...Permanent employmentImmediate start- ...technology relationships as a Researcher at Huawei Canada, focusing on agentic interactions. Participate in groundbreaking research that... ...for AI systems • Understanding of algorithms and prompt engineering • Ability to rapidly adapt to evolving AI tech • Enthusiasm for...Contract workWork at office
- We're hiring a contract AI engineer who builds, deploys and operates AI agents in production, using leading-edge agentic development practice to ship quickly. The AI Platform team builds AI agents and products for the trucking and fleet compliance space, used by both our...Contract workInternship
- We’re looking for a QA Automation Engineer who is passionate about software quality and automation to help ensure our products are reliable, scalable, and deliver an exceptional customer experienceToday, our automated test coverage is unit tests and some integration testing...Work at office
- The Role: Senior QA EngineerWhat you are looking for:A closely connected cultureA total rewards package meant to enhance your work... ...business and social causes through the Rexall Care NetworkThe Senior QA Engineer will lead the design and delivery of automated testing solutions...Shift work
- Job Title: Sr. QA Automation Engineer Location: Canada (Remote) Employment Type: Contract Job SummaryWe are looking for a skilled QA Automation Engineer to design, develop, and maintain automated test frameworks and scripts to ensure high-quality software...Contract workRemote work
$106k - $156k per year
...teamThe Huawei Human-Machine Interaction Lab unites global researchers, engineers, and designers to redefine human technology relationships through user centered, hands on research. We focus on agentic AI and multimodal interaction (voice, touch, vision, gesture) across smartphones...Contract workInternshipImmediate start- Modernize complex production software in Toronto across front-end, back-end, data, pipelines, integrations, and AI components.As an Agentic Engineer, you will contribute to a significant existing-platform and backend architecture rebuild. The position combines broad software...
- ...Cohere is seeking an Engineering Manager to lead our Deployment Engineering team in North America. This is a hands-on leadership role focused on technical excellence and delivering agentic AI integrations at scale for enterprise customers. You will mentor Forward Deployed...
$200k - $250k per year
Join Snaplii as a founding member of a small, agile engineering team that is building the way AI agents pay. Location: Etobicoke (Toronto), onsite with hybrid 3 days max · Compensation: CAD $200k–250k + 20% performance bonus + equity About SnapliiSnaplii is a...
Do you want to receive more vacancies?
Subscribe and receive similar vacancies to Agentic Qa Engineer. Be the first to apply!
Related searches
- software quality assurance engineer Winnipeg, MB
- software test engineer Winnipeg, MB
- qa engineer Winnipeg, MB
- senior quality assurance engineer Winnipeg, MB
- entry level qa Winnipeg, MB
- remote qa Winnipeg, MB
- pharmaceutical quality assurance Winnipeg, MB
- software qa Winnipeg, MB
- assurance qualité Winnipeg, MB
- analyste assurance qualité Winnipeg, MB

