Senior Quality Engineer - Test Automation, AI Testing & Java/Python

Quality Engineer  – Test Automation, Java/Python & GenAI

Roles and Responsibilities:

• Partner with DevOps,   Development, Product, and   QE teams to   design and execute   exploratory, automated,   and AI-focused   tests that ensure   product quality.

• Define quality strategies, test plans, test   cases, and test   data for functional, integration, system, regression, API, database, and   acceptance testing.

• Design, develop, and   maintain scalable test   automation frameworks using   Java, Python, or   Groovy.

• Build automated tests   for AI-enabled   features, including prompt   behavior, response accuracy, hallucination risks, safety, reliability, latency,   and regression coverage.

• Validate AI integrations, APIs, microservices, data   pipelines, model interactions, and third-party   services.

• Establish testing approaches   for both deterministic   and non-deterministic   AI behavior, including   evaluation datasets, expected   behavior, quality thresholds, and repeatability.

• Integrate automated testing   and AI quality   checks into CI/CD pipelines in   partnership with DevOps   teams.

• Use tools and   frameworks such as   Selenium, Cucumber/Gherkin, Playwright, RestAssured, JUnit, TestNG, ReadyAPI, and JMeter.

• Identify, document, prioritize, and track defects   through resolution.

• Analyze test coverage, quality trends, AI   evaluation results, and   production feedback to   identify gaps and   drive continuous improvement.

• Collaborate with cross-functional Agile teams   to deliver reliable, secure, and user-focused product features.

• Communicate test results, risks, quality metrics, and release recommendations   clearly to technical   and non-technical stakeholders.

Required Skills:

• 5+ years of   relevant experience in   Quality Engineering, Quality   Assurance, test automation, or software development.

• 5+ years of   programming experience   with Java,   Python, Groovy, or   a comparable language.

• 5+ years of   experience building and   maintaining test automation   frameworks.

• Hands-on experience   with Selenium, Cucumber/Gherkin, Playwright, RestAssured, or similar   automation technologies.

• Experience testing functional   and non-functional   requirements, APIs, integrations, databases, microservices, and regression suites.

• Experience with testing   frameworks and tools   such as JUnit, TestNG, qTest, ReadyAPI, and JMeter.

• Understanding of TDD, ATDD, BDD, continuous   testing, and risk-based testing practices.

•  Experience testing AI   or machine learning-powered applications, APIs, conversational systems, or generative AI   features.

• Understanding of AI   quality concepts such   as prompt evaluation, grounding, accuracy, relevance, toxicity, bias, hallucination detection, and   response consistency.

• Ability to create   effective test data   and evaluation datasets   for AI-driven   functionality.

• Strong analytical, problem-solving, communication,   and technical documentation   skills.

• Ability   to work effectively   in a fast-paced, cross-functional   Agile environment.

Nice to Have:

• Experience with AI   evaluation frameworks,   LLM testing, prompt   engineering, retrieval-augmented generation, embeddings, or model observability.

• Experience with Git, Bitbucket, Jenkins, CI/CD pipelines, Artifactory, SonarQube, Jira, Confluence, AWS, Kubernetes, Maven, or   Gradle.

• Experience testing cloud-native applications and   distributed microservice   architectures.

• Experience with security, performance, accessibility, or reliability testing.

• Experience developing   automation scripts and   reusable testing utilities.

• Experience   with Tableau or   other analytics and   visualization platforms.

• CSTE certification or   an equivalent quality   engineering certification.

• Experience working directly   with DevOps teams   to implement automated   quality gates and   release controls.