Senior Software Quality Engineer - AI

PeoplePilot

We are seeking a Senior Software Quality Engineer – AI Focused to drive quality, reliability, and automation for AI-enabled software products. The role will collaborate with Development, DevOps, Product, and QE teams to design robust testing strategies and scalable automation frameworks. The engineer will test AI/ML and Generative AI features, APIs, microservices, integrations, and data pipelines while ensuring reliable and secure product delivery. The position is full-time and onsite in Toronto, ON .

Key Requirements

5+ years of experience in Quality Engineering, QA, SDET, or Test Automation .
5+ years of programming experience with Java, Python, Groovy, or similar languages .
Strong experience building and maintaining test automation frameworks .
Hands-on experience with Selenium, Playwright, Cucumber/Gherkin, or RestAssured .
Experience with API, database, integration, regression, and microservices testing .
Experience with JUnit, TestNG, ReadyAPI, qTest, and JMeter .
Hands-on experience testing AI/ML, Generative AI, conversational AI, or LLM-based applications .
Knowledge of AI quality concepts such as prompt evaluation, accuracy, relevance, grounding, hallucination detection, toxicity, bias, and response consistency .
Experience with CI/CD, Agile, BDD, TDD, ATDD, and risk-based testing .
Strong analytical, problem-solving, communication, and documentation skills.

Key Responsibilities

Develop and execute comprehensive quality strategies, test plans, test cases, and test data .
Design, develop, and maintain scalable test automation frameworks .
Create automated tests for AI/ML and Generative AI features .
Evaluate AI responses for accuracy, relevance, safety, hallucination, reliability, and latency .
Test APIs, microservices, databases, integrations, data pipelines, and third-party services .
Create AI evaluation datasets, expected behaviors, quality thresholds, and regression coverage.
Integrate automated testing and AI quality checks into CI/CD pipelines .
Identify, document, prioritize, and track defects through resolution.
Analyze test coverage, quality metrics, AI evaluation results, and production feedback.
Collaborate with Development, DevOps, Product, and QE teams to improve product quality and release readiness.

#J-18808-Ljbffr

Apply

Vacancy posted 2 hours ago

Similar jobs that could be interesting for youBased on the Senior Software Quality Engineer - AI in Toronto, ON vacancy

Senior AI Software Engineer
$150k - $200k per year
...Stikeman Elliott is building its own agentic AI platform, with production agents working... ...and continues to expand. This is the senior engineering role responsible for advancing its... ...role that spans application architecture, software development and production delivery. You...
Senior
Full time
Stikeman Elliott Llp
Toronto, ON
10 days ago
AI Software Engineer
...usher in this new era, we seek AI-native thinkers across every... ...part of that mission. Our AI/ML engineering teams are delivering the... ...platform. We're looking for a Software Engineer — AI to join our growing... ...platform Build production-quality infrastructure for AI...
Senior
Full time
Work at office
Local area
Worldwide
Snowflake
Toronto, ON
10 days ago
Senior / Staff Software Engineer, AI Tooling
...Waabi, founded by AI visionary Raquel Urtasun, is the leader in Physical AI. With... ...from ML model development, to traditional software development, to non-technical business... .... We are looking for hands-on senior software engineers to build the AI tooling, agents, and workflows...
Senior
Full time
Work at office
Work from home
Flexible hours
Waabi
Toronto, ON
18 days ago
Software Engineer: Product Quality
$120k - $140k per year
...Description Position: Software Engineer: Product Quality Salary: $120,000 to $140,000 CAD Location: Canada based. About Fiscal.ai Fiscal.ai is a leading research and data platform built for capital markets professionals. Combining a high-performance research...
Suggested
Full time
Immediate start
Shift work
Fiscal.ai
Toronto, ON
24 days ago
Senior Software Engineer, AI
...Building the Future of Competitive Intelligence Klue Engineering is hiring! We're looking for a Senior Software Engineer to join our team in Toronto, focusing on... ..., and human-in-the-loop) that measure relevance, quality, latency, and end-to-end task success across our...
Senior
Full time
Work at office
Klue Inc.
Toronto, ON
more than 2 months ago
Senior Software Engineer, AI
$167.5k - $235k per year
...cross-functional team to build AI agents that fundamentally... ...scale. Instead of writing bespoke software for every workflow, we are developing... ...consists of fewer than five engineers. The work spans critical areas... .... We are seeking a Senior Software Engineer with a strong...
Senior
Long term contract
Full time
Casual work
Internship
Work at office
Local area
Immediate start
Justworks
Toronto, ON
more than 2 months ago
Senior Software Engineer
...performant, and aligned with Sitetracker's engineering standards. This opportunity offers the... ...technical design decisions, implementation quality, and architectural integrity while ensuring... ...development frameworks and effectively used AI-assisted development tools to accelerate...
Senior
Long term contract
Full time
Sitetracker
Toronto, ON
7 days ago
Senior Software Engineer, Applied AI & Customer Solutions
$137k - $172k per year
...customers build the skills they need for a world being reshaped by AI. Read more about the combined company by visiting our blog... ...make a difference at global scale. Job Overview As a Senior Software Engineer, you will join a fast-paced innovation team that builds and...
Senior
Full time
Live In
Remote work
Coursera
Toronto, ON
17 days ago
Senior Software Engineer - Applied AI
...expanding enterprise demand, we’re scaling our engineering team to support increasingly complex, data... ...We’re Hiring Feathery is building an AI-native product experience — from a... ...etc.) and optimize for cost, latency, and quality Develop asynchronous pipelines using AWS...
Senior
Work at office
Feathery
Toronto, ON
a month ago
Senior Software Engineer - Canada
...About Canals Canals builds software for wholesale distributors, helping them operate more efficiently through automation and AI. Our customers are the companies responsible... ...at pace. To keep up, we need senior-level engineers who can move quickly while maintaining...
Senior
Remote job
Full time
Flexible hours
Canals
Toronto, ON
19 hours ago
Senior Software Engineer, AI Entries
$136k - $170k per year
...opportunity to thrive. Lyft is looking for software engineers from a scope of disciplines. We are... ...passengers and drivers. The Applied AI team is looking for a Backend Engineer to... ...in code reviews to ensure code quality and share knowledge across the team Participate...
Senior
Hourly pay
Full time
Work at office
Flexible hours
3 days per week
Lyft
Toronto, ON
more than 2 months ago
Senior AI Software Developer
$125k - $175k per year
...seeking to recruit the best and the brightest talents. The Senior AI Software Developer will design, build, and deliver the firm’s AI... ...plugins, memory, scheduled and headless execution, and context engineering. Develop and maintain MCP servers and tool sets, including...
Senior
Full time
Work at office
Stikeman Elliott Llp
Toronto, ON
23 days ago
Software Engineer, Applied AI
...About the Role SJC Media is building the AI layer for its Digital Operations. As the founding Software Engineer, Applied AI, you own that layer end-to-end: from... ...systems that are reliable, secure, and auditable. Senior engineering ability: Write clean, documented code...
Senior
Full time
Flexible hours
Urban Ridge Supplies
Toronto, ON
28 days ago
Senior Software Engineer, International
$166k - $195k per year
...are the rewards. The International engineering team's mission is to expand Robinhood... ...their financial lives! As a Senior Full Stack Software Engineer , you will be a core technical... ...Engineering: Develop and maintain high-quality frontend interfaces (React/TypeScript)...
Senior
Full time
Work at office
Worldwide
Flexible hours
Shift work
3 days per week
Robinhood
Toronto, ON
9 days ago
Senior Software Engineer
$136k - $187k per year
...Secure Every Identity, from AI to Human Identity is the key to unlocking the potential of AI. Okta secures AI by building... ...this mission. If you are too, let's talk. We are looking for a Senior Software engineer who is passionate about writing the tools to integrate and...
Senior
Full time
Local area
Worldwide
Flexible hours
Okta
Toronto, ON
25 days ago
Senior Software Engineer - Open Source
$120 per hour
...Mercor connects elite creative and technical talent with leading AI research labs. Headquartered in San Francisco, our investors... ..., Larry Summers , and Jack Dorsey . Position: Senior Software Engineer Type: Contract Compensation: $120/hour...
Senior
Full time
Contract work
Summer work
Remote work
Mercor
Toronto, ON
3 days ago
Senior Software Engineer - Full Stack
...What you'll do: ~6+ years of software development experience,... ...Takes personal responsibility for quality and maintainability of the product... .../knowledge base and team/engineering presentations ~ Strong ability... ...use artificial intelligence (AI) tools to support parts of the...
Senior
Full time
Emburse
Toronto, ON
5 days ago
Senior Software Engineer, iOS
$137.2k - $196k per year
...About the Role: As a Senior iOS Engineer at Tubi, you’ll play a critical role on our mobile engineering... ...solutions. Ensure the performance, quality, and responsiveness of applications are... ...and ad integration. Leverage AI-powered tools and automation solutions to...
Senior
Long term contract
Remplacement
Full time
Temporary work
Local area
Flexible hours
Tubi, Inc.
Toronto, ON
22 days ago
Senior Software Development Engineer in Test (SDET), QA - AI Systems
...About Nexxa Nexxa is building the best AI systems for heavy industries,... ...Role Overview We’re looking for a Senior / Staff SDET to own quality for Nexxa’s AI agent systems, combining... ...thing. You’ll work closely with ML engineers, backend engineers, and Forward Deployed...
Senior
Full time
Contract work
Internship
Shift work
Nexxa
Toronto, ON
3 days ago
Senior Software Engineer II, Investors
$181.05k - $213k per year
...is the connected platform and AI-native ecosystem for private capital... ..., Carta brings together the software, services, and legal... ...reporting. You’ll join small, senior teams that blend startup-style... ...money. We’re hiring multiple engineers, and you’ll be hired into the...
Senior
Full time
Internship
Work at office
1 day per week
Dba Carta, Inc.
Toronto, ON
7 days ago
Senior Software Engineer in Test
$118k - $162k per year
...Secure Every Identity, from AI to Human Identity is the key to unlocking the potential... ...across thousands of applications. Quality here is a compliance bar, not just a user... ...auditor evidence that isn't true. The Senior Software Development Engineer in Test Opportunity...
Senior
Full time
Local area
Worldwide
Okta
Toronto, ON
24 days ago
Senior Quality Engineer
$157.5k - $192.5k per year
...moving securely and predictably. Core and Risk Engineering are tied at the hip, and you'll be... ..., and uptime. Your mission is to scale quality operations and build the tooling, frameworks... ...make the numbers mean something. You are AI-fluent in practice — using AI tools daily...
Senior
Full time
Internship
Local area
Shift work
Relay
Toronto, ON
a month ago
Staff Software Engineer, Quality & Reliability
$172k - $229k per year
...BuildOps is looking for a Staff Software Engineer to set and drive our company-wide technical strategy... ...functional role for an engineer who sees quality and reliability as properties of the... ...commitment to equipping contractors with AI-driven tools to conquer chaos, boost efficiency...
Long term contract
Permanent employment
Full time
For contractors
Work at office
Local area
Work from home
Flexible hours
Buildops
Toronto, ON
more than 2 months ago
Software Engineer, AI
$90k per year
...cutting-edge cloud platform, coupled with AI-driven analytics tools, unlocks the true... ...looking for a highly skilled and experienced Software Engineer, AI to perform a key role in our digital... ...memory/context management. Improve quality and evaluation Define offline and online...
Full time
Flexible hours
Calabrio, Inc.
Toronto, ON
more than 2 months ago
AI Product Quality Specialist
$15 - $30 per hour
...us forward. Across strategy, engineering, design, data, and operations,... ...explore, design, and implement AI strategies that are secure, scalable... .... The Opportunity Our senior team is shipping fast. Too... ...the person who keeps product quality tight so leadership can stay...
Senior
Remote job
Hourly pay
Full time
Contract work
For contractors
Manual labor
Immediate start
Flexible hours
Human Agency Inc
Toronto, ON
25 days ago
Senior Software Engineer - Product Security
$160k - $222k per year
...join ours. About The Role Our Engineering organization owns the software that makes our marketplace work. Our... ...listen to the data; and iterate. As a Senior Software Engineer, Product Security,... ..., including the latest enterprise AI tools, to help you work smarter and get...
Senior
Full time
Work at office
Local area
Remote work
Monday to friday
3 days per week
Faire
Toronto, ON
4 days ago
Software engineer senior H/F (IT)
...foundations. They currently have an opening on their team to add a Senior Software Engineer. The Senior Software Engineer will be responsible for... ...Process/Atom Logs). Own development end to end from high-quality coding, unit testing, UAT support to Prod deployment. Hands...
Senior
Full time
Hire DigITalent Inc.
Toronto, ON
a month ago
Senior Software Engineer, Observability (Auth0)
$136k - $187k per year
...Secure Every Identity, from AI to Human Identity is the key... ...looking for an Observability Engineer to help ensure our Product and... ...while we continue to rapidly ship software our customers love. Our... ...development in these areas. As a Senior Engineer on this team, you...
Senior
Full time
Local area
Worldwide
Okta
Toronto, ON
8 days ago
Senior Software Engineer, Foundational Storage
$200k - $420k per year
...About Sentry Software runs the world and the pace is faster than ever... ...and our team is building its AI-native future. About the... ...stores, and caches. We help Sentry engineers store and access data... ...instance holds their data. As a Senior Software Engineer on the team,...
Senior
Hourly pay
Full time
Sentry
Toronto, ON
9 days ago
Senior Software Engineer, Security Engineering (Auth0)
$136k - $187k per year
...Secure Every Identity, from AI to Human Identity is the key to unlocking the potential of AI. Okta secures AI by building... .... We're all in on this mission. If you are too, let's talk. Senior Software Engineer, Security Engineering We are looking for a Senior Software...
Senior
Full time
Local area
Worldwide
Okta
Toronto, ON
23 days ago

Do you want to receive more vacancies?

Subscribe and receive similar vacancies to Senior Software Quality Engineer - AI. Be the first to apply!