Senior Quality Engineer - Test Automation, Ai Testing & Java/Python
Astra-North Infoteck Inc.
Senior Quality Engineer – Test Automation, AI Testing & Java/PythonQuality Engineer– Test Automation, Java/Python & GenAI
Roles and Responsibilities:
Partner with DevOps,Development, Product, andQE teams todesign and executeexploratory, automated,and AI-focusedtests that ensureproduct quality.
Define quality strategies, test plans, testcases, and testdata for functional, integration, system, regression, API, database, andacceptance testing.
Design, develop, andmaintain scalable testautomation frameworks usingJava, Python, orGroovy.
Build automated testsfor AI-enabledfeatures, including promptbehavior, response accuracy, hallucination risks, safety, reliability, latency,and regression coverage.
Validate AI integrations, APIs, microservices, datapipelines, model interactions, and third-partyservices.
Establish testing approachesfor both deterministicand non-deterministicAI behavior, includingevaluation datasets, expectedbehavior, quality thresholds, and repeatability.
Integrate automated testingand AI qualitychecks into CI/CD pipelines inpartnership with DevOpsteams.
Use tools andframeworks such asSelenium, Cucumber/Gherkin, Playwright, RestAssured, JUnit, TestNG, ReadyAPI, and JMeter.
Identify, document, prioritize, and track defectsthrough resolution.
Analyze test coverage, quality trends, AIevaluation results, andproduction feedback toidentify gaps anddrive continuous improvement.
Collaborate with cross-functional Agile teamsto deliver reliable, secure, and user-focused product features.
Communicate test results, risks, quality metrics, and release recommendationsclearly to technicaland non-technical stakeholders.
Required Skills:
5+ years ofrelevant experience inQuality Engineering, QualityAssurance, test automation, or software development.
5+ years ofprogramming experiencewith Java,Python, Groovy, ora comparable language.
5+ years ofexperience building andmaintaining test automationframeworks.
Hands-on experiencewith Selenium, Cucumber/Gherkin, Playwright, RestAssured, or similarautomation technologies.
Experience testing functionaland non-functionalrequirements, APIs, integrations, databases, microservices, and regression suites.
Experience with testingframeworks and toolssuch as JUnit, TestNG, qTest, ReadyAPI, and JMeter.
Understanding of TDD, ATDD, BDD, continuoustesting, and risk-based testing practices.
Experience testing AIor machine learning-powered applications, APIs, conversational systems, or generative AIfeatures.
Understanding of AIquality concepts suchas prompt evaluation, grounding, accuracy, relevance, toxicity, bias, hallucination detection, andresponse consistency.
Ability to createeffective test dataand evaluation datasetsfor AI-drivenfunctionality.
Strong analytical, problem-solving, communication,and technical documentationskills.
Abilityto work effectivelyin a fast-paced, cross-functionalAgile environment.
Nice to Have:
Experience with AIevaluation frameworks,LLM testing, promptengineering, retrieval-augmented generation, embeddings, or model observability.
Experience with Git, Bitbucket, Jenkins, CI/CD pipelines, Artifactory, SonarQube, Jira, Confluence, AWS, Kubernetes, Maven, orGradle.
Experience testing cloud-native applications anddistributed microservicearchitectures.
Experience with security, performance, accessibility, or reliability testing.
Experience developingautomation scripts andreusable testing utilities.
Experience with Tableau orother analytics andvisualization platforms.
Experience working directlywith DevOps teamsto implement automatedquality gates andrelease controls.
#J-18808-Ljbffr
Roles and Responsibilities:
Partner with DevOps,Development, Product, andQE teams todesign and executeexploratory, automated,and AI-focusedtests that ensureproduct quality.
Define quality strategies, test plans, testcases, and testdata for functional, integration, system, regression, API, database, andacceptance testing.
Design, develop, andmaintain scalable testautomation frameworks usingJava, Python, orGroovy.
Build automated testsfor AI-enabledfeatures, including promptbehavior, response accuracy, hallucination risks, safety, reliability, latency,and regression coverage.
Validate AI integrations, APIs, microservices, datapipelines, model interactions, and third-partyservices.
Establish testing approachesfor both deterministicand non-deterministicAI behavior, includingevaluation datasets, expectedbehavior, quality thresholds, and repeatability.
Integrate automated testingand AI qualitychecks into CI/CD pipelines inpartnership with DevOpsteams.
Use tools andframeworks such asSelenium, Cucumber/Gherkin, Playwright, RestAssured, JUnit, TestNG, ReadyAPI, and JMeter.
Identify, document, prioritize, and track defectsthrough resolution.
Analyze test coverage, quality trends, AIevaluation results, andproduction feedback toidentify gaps anddrive continuous improvement.
Collaborate with cross-functional Agile teamsto deliver reliable, secure, and user-focused product features.
Communicate test results, risks, quality metrics, and release recommendationsclearly to technicaland non-technical stakeholders.
Required Skills:
5+ years ofrelevant experience inQuality Engineering, QualityAssurance, test automation, or software development.
5+ years ofprogramming experiencewith Java,Python, Groovy, ora comparable language.
5+ years ofexperience building andmaintaining test automationframeworks.
Hands-on experiencewith Selenium, Cucumber/Gherkin, Playwright, RestAssured, or similarautomation technologies.
Experience testing functionaland non-functionalrequirements, APIs, integrations, databases, microservices, and regression suites.
Experience with testingframeworks and toolssuch as JUnit, TestNG, qTest, ReadyAPI, and JMeter.
Understanding of TDD, ATDD, BDD, continuoustesting, and risk-based testing practices.
Experience testing AIor machine learning-powered applications, APIs, conversational systems, or generative AIfeatures.
Understanding of AIquality concepts suchas prompt evaluation, grounding, accuracy, relevance, toxicity, bias, hallucination detection, andresponse consistency.
Ability to createeffective test dataand evaluation datasetsfor AI-drivenfunctionality.
Strong analytical, problem-solving, communication,and technical documentationskills.
Abilityto work effectivelyin a fast-paced, cross-functionalAgile environment.
Nice to Have:
Experience with AIevaluation frameworks,LLM testing, promptengineering, retrieval-augmented generation, embeddings, or model observability.
Experience with Git, Bitbucket, Jenkins, CI/CD pipelines, Artifactory, SonarQube, Jira, Confluence, AWS, Kubernetes, Maven, orGradle.
Experience testing cloud-native applications anddistributed microservicearchitectures.
Experience with security, performance, accessibility, or reliability testing.
Experience developingautomation scripts andreusable testing utilities.
Experience with Tableau orother analytics andvisualization platforms.
Experience working directlywith DevOps teamsto implement automatedquality gates andrelease controls.
#J-18808-Ljbffr
Vacancy posted 1 hour ago
Similar jobs that could be interesting for youBased on the Senior Quality Engineer - Test Automation, Ai Testing & Java/Python in Winnipeg, MB vacancy
- ...time professionals, and our annual growth rate exceeds 25%. Job Description Miratech is looking for a skilled Senior AI Engineer with Python, LangGraph and LangChain to join our US healthcare client. You will play a key role in developing and enhancing an enterprise...SeniorPythonFull timeInternshipRemote workRelocation
- ...largest Indigenous-owned software testing and technology services company... ...who are passionate about quality, teamwork, and making an impact... ...looking for experienced Test Automation Specialists who are passionate about modern engineering practices and excited about how...PythonJavaInternship
- Engineer Python automation for EA-SPORTS FC quality verification at Electronic Arts, integrating CI/CD diagnostics and machine learning. This is hybrid work in Burnaby, BC.As a Software Engineer III, you will build tools that expand test coverage and detect defects earlier....Python
- Build AI test automation with Java or Python as a Senior Quality Engineer focused on prompt behavior, accuracy, and reliable releases.This senior position requires five-plus years in quality engineering, quality assurance, test automation, or software development. You will create...SeniorPythonJava
- Become a key part of our team as an AI Quality Engineer, focusing on automated test development and quality assurance. This role combines Python expertise with a deep understanding of AI processes. With a minimum of 4 years in QA or test automation, you'll build and maintain...Python
- Build AI test automation at CrowdStrike as a Software Development Engineer in Test, working remotely with Python and distributed systems.This hands-on role supports the Detonations and Threat Analysis Platform Engineering team. You will develop functional, integration, and performance...SeniorPythonJavaRemote work
- Lead AI chatbot validation through Java, Python, and scalable automation frameworks while partnering with DevOps teams on CI/CD quality checks.This Senior Quality Engineer / SDET role supports DevOps enablement by delivering exploratory, automated, and AI-focused testing. You...SeniorPythonJava
- ...for a hybrid position (4 days a week). The role requires 8-10+ years of experience, strong programming skills in Java/Python, and expertise in test automation frameworks. Key responsibilities include designing automation solutions, collaborating with teams to validate integrations...SeniorPythonJava
- Software quality and test automation define this remote-first Quality Engineer role at StackAdapt, open to candidates anywhere in Canada.StackAdapt seeks a Quality Engineer for a cross-functional agile environment supporting successful product releases. You will build reliable...PythonRemote work
- AI test automation and software quality define this hands-on specialist role at PLATO, supporting modern engineering practices across Canada’s 100% Canada-based team.PLATO seeks experienced Test Automation Specialists with five or more years in testing, automation, development...SeniorPythonJava
- Build smarter Playwright tests with GitHub Copilot, CodeGen, and MCP tools in Veriday Inc.'s remote Senior Automation Engineer contract.This six-month Canadian engagement, potentially extendable, requires hands-on engineering judgment beyond generated test output. You will...SeniorContract workRemote work
- Upgrade is seeking a Senior QA Automation Developer to own the end-to-end quality of a new product. You will build automated tests and perform functional testing to ensure a seamless banking experience for our customers. You will collaborate with product teams, define testing...SeniorJava
- ...define qualityengineering and test strategy across Agile... ...Developers, and Scrum teams to ensure quality delivery Ensure quality... ...experience insoftware testing/quality engineering Proven experience leading... ...hands-on experience withtest automation tools and frameworks...
- Design test automation and code with Python, Java, JavaScript, or TypeScript at SpryPoint, building confidence across critical utility software workflows.As a Senior SDET, you will create automation foundations for a new team and influence testing standards across product squads...SeniorPythonJava
- Build automated testing with Selenium, Parasoft, and Robot Framework while delivering in-sprint quality within Agile development teams.This Test Automation Engineer develops, maintains, and executes automated tests across platforms and technologies. The work covers database...SeniorPython
- Take on a pivotal role as an AI Quality Assurance and Testing Engineer, focusing on the assessment of machine learning and generative AI models. Ensure high standards of quality, bias assessment, and responsible AI practices across applications. In this short-term contract...PythonTemporary work
- Build software quality at StackAdapt as a remote Senior Test Automation Engineer, using Playwright, API testing, and CI/CD automation across North America.This senior position supports product releases in a cross-functional agile environment. You will develop reliable automated...SeniorPythonRemote work
- Shape trusted AI evaluation at CrowdStrike using agent frameworks, LLM APIs, and verification-first test automation. Work on cloud-based cybersecurity systems.As a Software Development Engineer in Test, you will help determine how CrowdStrike AI agents are tested, evaluated...Python
- ...Senior Automation Engineer (Project Manager) Saguenay-Lac-Saint-Jean, QC, Canada Type of position: Regular To pursue project and client... ...Help prepare technical and commercial proposals Do your qualities and values match our corporate culture? Results‑driven...SeniorLong term contractTemporary workWork at officeLocal area
- Engineer CMOD automation and integrations for CGI’s Banking Services Division from Downtown Toronto, with four office days and one remote day.As a Senior Java Python Developer, you will enhance IBM CMOD applications and enterprise content lifecycle management. The position emphasizes...SeniorPythonJavaWork at officeRemote work1 day per week
- Lead enterprise Test Automation and DevOps initiatives as a Senior Test Manager in a hybrid Toronto or Kitchener office arrangement. Shape quality engineering across complex insurance and retirement systems.This senior leadership position bridges technical execution with business...SeniorJavaWork at officeShift work
- Lead automation testing with Selenium, Parasoft, and Robot Framework in a senior SDET role requiring six-plus years of experience.This SDET position focuses on automating tests... ...development lifecycle. Apply expertise in C#, Python, or Java alongside continuous integration tools...SeniorPythonJava
- Build reliable AI product testing as a Senior Software Developer in Test with Vena, working remotely from Canada on automated quality solutions.This full-time senior role supports Vena’s agentic AI solutions within Product Development. You will develop, execute, and maintain...SeniorFull timeRemote work
- ...position is listed on behalf of a partner company, who manages all applications and next steps. Our partner is looking for a Senior AI Engineer (Python) based in Canada. Join a high-performing engineering team to develop and enhance an enterprise AI platform that powers...SeniorPythonRemote workRelocationRelocation package
$81.2k - $122k per year
Senior Automation Test Engineer – VancityVancity is a member‑owned credit union committed to inclusion and social justice. We are looking for a Senior Automation Test Engineer to join our 2,700‑person team. About the workplaceThis is a permanent, full‑time role that offers...SeniorPermanent employmentFull timeTemporary workWork at officeFlexible hours- AI-driven test automation defines this Test Automation Specialist role at PLATO, focused on scalable frameworks, modern engineering, and practical AI applications.PLATO seeks experienced Test Automation Specialists with five-plus years in testing, automation, development, or...PythonJava
- Lead Java quality engineering strategy for high-throughput trading platforms in Mississauga, ON, with expertise in Spring Boot and microservices... ...is remote, with potential hybrid work after conversion.As a Senior Forward Deployed Java Engineer in Test, you will define global...SeniorPythonJavaPermanent employmentContract workRemote work
- Build software quality through test automation and CI/CD engineering as a Senior SDET in Calgary, working in a hybrid model.Reporting to the Software Development Manager, this senior engineering role develops maintainable automation across APIs, integrations, and web applications...SeniorJavaContract work
- Become a Senior Mobile Test Automation Engineer, focusing on mobile application quality, either in Toronto or remote. Leverage automation to ensure exceptional performance and reliability. We are looking for an experienced QE/Test Automation Engineer with a robust background...SeniorJavaRemote work
- Build software quality through test automation, C#, and CI/CD pipelines as a Senior SDET in Calgary’s hybrid engineering environment. H&R Block supports technical quality leadership.Reporting to the Software Development Manager, this senior engineering role develops maintainable...SeniorJavaContract work
Do you want to receive more vacancies?
Subscribe and receive similar vacancies to Senior Quality Engineer - Test Automation, Ai Testing & Java/Python. Be the first to apply!
Related searches
- quality engineer Winnipeg, MB
- ingénieur qualité Winnipeg, MB
- technicien qualité Winnipeg, MB
- quality engineer manufacturing Winnipeg, MB
- quality engineer- qms Winnipeg, MB
- entry level quality engineer Winnipeg, MB
- test automation specialist Winnipeg, MB
- automation application engineer Winnipeg, MB
- control and automation engineer Winnipeg, MB
- electrical automation engineer Winnipeg, MB

