QA Engineer - Machine Learning

ai is on a mission to democratize AI. As the world’s leading agentic AI company, H2O.ai converges Generative and Predictive AI to help enterprises and public sector agencies develop purpose-built GenAI applications on their private data. With a focus on Sovereign AI—secure, compliant, and infrastructure-flexible deployments—H2O.ai delivers solutions that align with the highest standards of data privacy and control.
Our open-source technology is trusted by over 20,000 organizations worldwide, including more than half of the Fortune 500. H2O.ai powers AI transformation for companies like AT&T, Commonwealth Bank of Australia, Chipotle, Workday, Progressive Insurance, and NIH.
ai partners include NVIDIA, Dell Technologies, Deloitte, Ernst & Young (EY), Snowflake, AWS, Google Cloud Platform (GCP), VAST Data and MinIO. H2O.ai’s AI for Good program supports nonprofit groups, foundations, and communities in advancing education, healthcare, and environmental conservation. With a vibrant community of 2 million data scientists worldwide, H2O.ai aims to co-create valuable AI applications for all users.
ai has raised $256 million from investors, including Commonwealth Bank, NVIDIA, Goldman Sachs, Wells Fargo, Capital One, Nexus Ventures and New York Life.
You will be a core QA engineer responsible for the end-to-end quality and reliability of h2o.ai's product portfolio .
This is a remote position, with a preference for candidates based in Canada within the PST timezone. Design, build, and maintain Python-first automation frameworks (pytest + Playwright + async) for UI, API, and end-to-end workflows of h2o.ai's product portfolio .
Perform deep exploratory and manual testing of the h2o.ai's product portfolio web UI (chat interfaces, document upload/processing, agent builder, maker suite, evaluation hub, enterprise admin console).
ai's product portfolio itself (and other GenAI tools) in creative ways to:
Generate realistic test documents, datasets, and edge-case prompts.
Auto-generate or refine test cases via prompt engineering.
Root-cause of difficult, intermittent failures in distributed RAG/LLM systems by combining traditional log analysis with GenAI-assisted debugging.
Create and execute chaos experiments targeting LLM routing, vector database latency, GPU OOM, retrieval failures, and token-limit edge cases.
ai's product portfolio clusters on Kubernetes for testing (Helm, custom operators).
Own UI regression suites (Playwright) and accessibility testing.
Write reproducible, high-quality bug reports that developers love and regularly verify fixes across the full stack.
ai's product portfolio feature teams in an Agile environment and influence testability from the design phase.
2-4 years of QA experience with a strong mix of automation and hands-on manual/exploratory testing.

• Decent Python skills and experience building maintainable test frameworks from scratch.
• Real-world experience testing modern React/TypeScript web applications and writing bulletproof Playwright or Selenium tests.
• Hands-on Kubernetes experience in a testing or test-environment context (kubectl, Helm, writing manifests, debugging pods).
• Proven ability to use generative AI tools daily to accelerate debugging, test-data creation, and log analysis (you’ve already used h2o.ai's product portfolio , ChatGPT, Claude, or similar in your current QA workflow).
• Solid grasp of CI/CD (GitHub Actions preferred) and infrastructure-as-code concepts.

Prior experience testing RAG systems, agentic workflows, or enterprise chat/assistant platforms.
Chaos engineering on Kubernetes (Chaos Mesh, Litmus) or GPU workloads.
Familiarity with Chaos engineering principles.
Basic understanding of containerization (Docker/Kubernetes concepts like pods and kubectl) in a testing context.
Why H2O.ai?
Remote-friendly culture
Flexible working environment

80,000 - $95,000 CAD

ai is committed to creating a diverse and inclusive culture. All qualified applicants will receive consideration for employment without regard to their race, ethnicity, religion, gender, sexual orientation, age, disability status or any other legally protected basis.
ai is an innovative AI cloud platform company, leading the mission to democratize AI for everyone. Thousands of organizations from all over the world have used our cutting-edge technology across a variety of industries. We push the boundaries of what is possible with artificial intelligence.
ai employs the world’s top Kaggle Grandmasters, the community of best-in-the-world machine learning practitioners and data scientists. A strong AI for Good ethos and responsible AI drive the company’s purpose.