Software Development Engineer in Test - GenAI / LLM
Full-time
Astra North Infoteck Inc.
Software Development Engineer in Test – GenAI / LLM
Role Description Work Arrangement: 1 day onsite per week
Experience Required: 6–8 years
Key Roles & Responsibilities
- Validate chatbot responses for accuracy, relevance, completeness, and consistency.
- Create and execute test scenarios, test cases, and regression test suites.
- Identify hallucinations, incorrect responses, formatting issues, and knowledge gaps.
- Perform chatbot conversation analysis and recommend quality improvements.
- Develop and maintain test automation for chatbot validation where possible.
- Work closely with developers, AI engineers, and content teams to resolve defects.
- Track, document, and report defects, test results, and quality metrics.
- Support UAT, release validation, and production quality monitoring.
Key Skillsets
- 5 years of strong QA SDET test automation experience.
- Experience testing AI, LLM, GenAI, chatbots, virtual assistants, or conversational platforms.
- Knowledge of prompt engineering concepts.
- Experience with Jira, Confluence, GitHub, and Agile delivery.
- Experience with API testing and automation frameworks.
- Strong analytical and troubleshooting skills.
- Experience with Python, Selenium, Playwright, PyTest, or similar tools.
- Strong communication and documentation skills.
Essential Skills
- QA SDET and test automation.
- AI, LLM, GenAI, chatbots, and conversational platforms.
- Prompt engineering concepts.
- API testing and automation frameworks.
- Python, Selenium, Playwright, or PyTest.
- Jira, Confluence, GitHub, and Agile delivery.
- Analytical and troubleshooting skills.
- Communication and documentation skills.
Desirable Skills
- Digital: Python
- Digital: Chatbots / Conversational Agents
- Generative AI – Quality Engineering
Vacancy posted 19 hours ago
Similar jobs that could be interesting for youBased on the Software Development Engineer in Test - GenAI / LLM in Toronto, ON vacancy
- ...Join Episode Six today. What we’re hiring for: We are a growing fintech organization looking to add a Senior Software Development Engineer in Test! What you’ll be responsible for in this role: Leading testing effort in the payments, card issuing and revolving...SuggestedFull time
Episode Six
Toronto, ONmore than 2 months ago $118k - $162k per year
...can stay over-provisioned on access it believes it revoked, or hand an auditor evidence that isn't true. The Senior Software Development Engineer in Test Opportunity As a Senior Software Development Engineer in Test, you'll own quality for medium-sized features on your...SuggestedFull timeLocal areaWorldwideOkta
Toronto, ON13 days ago$75k - $95k per year
...committed to making a positive impact on its customers, employees, and communities. The Role We're hiring Associate Software Engineers in Test to work at Veeva, working on various products in the Life Sciences industry encompassing the fields of pharmaceuticals,...SuggestedFull timeInternshipWork at officeLocal areaWork from homeRelocationVeeva Systems
Toronto, ON10 days ago- ...living a healthier life, then you must contact us. \ \ \ Software Engineer in Test \\(Perm "Toronto - REMOTE \")\\\ \ This is a central... ...automated tests \ \ o \\\ Report bugs and errors to development teams \ \ o \\\ Track and report testing activities...SuggestedPermanent employmentFull timeRemote work
Yoush Consulting
Toronto, ONmore than 2 months ago $72k - $75k per year
...they love. The Role We're looking for an experienced Software Engineer in Testing to join our growing Engineering team! If you have a sharp... ...a key role in embedding quality into every stage of the development lifecycle. You’ll influence automation initiatives, refine...SuggestedFull timeInternshipWork at officeRemote workPush Operations
Toronto, ONmore than 2 months ago$118k - $162k per year
...to do career-defining work. We're all in on this mission. If you are too, let's talk. Okta is seeking an experienced Senior Software Engineer in Test for our UI Engineering team. As a senior member of product quality engineering, you will own the test framework and quality...Local areaWorldwideOkta
Toronto, ON11 days ago$137.6k - $172k per year
...difference at global scale. Job Overview: As a Senior Software Test Engineer (Test Infrastructure and Tooling) at Coursera, you’ll play a... ...automation (Github actions, Circleci, Kubernetes) ~ Experience with development and maintenance of test automation tools using popular open...Coursera
Toronto, ONa month ago- ...2024 | Best Workplaces™ for Professional Development 2025 | Best Workplaces™ for Young Talent... ...systems, partnering closely with Engineering leadership and the Website & Digital Marketing... ...Contribute to code quality through reviews, testing, and documentation CMS and content...Long term contractPermanent employmentFull timeImmediate startFlexible hours
Picton Mahoney Asset Management
Toronto, ONmore than 2 months ago $130k - $140k per year
...Description Are you a passionate Senior Software Engineer looking to make a significant impact in... ...will include: Front-End Development: Build responsive and dynamic user interfaces... ...enforce best practices for code quality, testing and deployment Cloud Integration:...Part timeWork at officeRemote workWork from homeFlexible hoursCurinos Inc
Toronto, ON2 days ago$140k - $192.5k per year
...backend to and grant proper application access.c) The Staff Software Engineer in Test Opportunity We are looking for an experienced Software... ...Mentor and technically guide engineers across quality and development teams; write and review designs and code in Swift with colleagues...Full timeLocal areaWorldwideOkta
Toronto, ONmore than 2 months ago- ...experimental mindset — who rapidly test emerging capabilities to... ...part of that mission. Our AI/ML engineering teams are delivering the... ...platform. We're looking for a Software Engineer — AI to join our growing... ...serving, inference pipelines, LLM integration, embedding systems...Full timeWork at officeLocal areaWorldwide
Snowflake
Toronto, ON8 hours ago - ...Emburse is seeking a skilled and detail-oriented Software Development Engineer in Test (SDET) to join our engineering team. In this role, you will bridge the gap between software development and quality assurance, designing and implementing robust automated testing frameworks...Full time
Emburse
Toronto, ONmore than 2 months ago - ...are seeking a versatile and experienced engineer to join our Inference Core Model Bringup... ...comfortable working across the entire Cerebras software stack. Your work will play a critical... .... Proven experience in compiler development, particularly with LLVM and/or MLIR....Full time
Cerebras Systems
Toronto, ONmore than 2 months ago - ...Senior Software Engineer – GenAI, AWS, Python & Java Skills and Responsibilities • Bachelor’s degree in computer science or a relevant field... ...• Hands-on expertise with Python, Lambda functions, APIs, LLM Models, and Java. • Experience with DevOps culture, tools...Permanent employmentCasual work
$100k per year
...paradigm, solutions must evolve to unify innovations in software models, compilers, platforms, networking, and... ...for contributors of all seniorities. As a Software Engineer on the Acceleration Kernel Development team at Tenstorrent, you’ll work at the intersection of...Permanent employmentFull timeTenstorrent
Toronto, ONmore than 2 months ago- ..., committed to making a positive impact on its customers, employees, and communities. The Role We're hiring Associate Software Engineers in Test to work at Veeva, working on various products in the Life Sciences industry encompassing the fields of pharmaceuticals, biotechnology...Full timeWork at officeWork from home
$175k - $210k per year
...Information: Job Title: Principal Software Engineer - Zynque Location: Toronto (Onsite/... ...supply chain and logistics. After a year of development, it is live in production with active... ...engineering: profiling, load testing, horizontal scaling, capacity planning...Long term contractFull timeInternshipLive InWork at officeImmediate startRemote workFlexible hoursFulfillment IQ
Toronto, ON17 days ago- ...3+ years of non-internship professional software development experience ~ Experience programming with... ...for the design, development, test, and deployment of a range of products that... ...technologies. If you are a software engineer with a creative flare, problem-solving skills...Full timeTemporary workInternshipSeasonal workLocal areaWorldwide
$45 - $55 per hour
...getting started. Role Overview As a Software Engineer, Intern (Winter 2027) on the Engineering... ...learn how modern application-layer and GenAI products are built. You’ll gain hands-on... ...a project end-to-end from coding and testing to deploying it to production, and presenting...InternshipWork at officeRelocation packageHarvey, Inc.
Toronto, ON10 days ago$160k - $220k per year
...talk. Opportunity As a Staff Engineer on the OIG team, you will be involved into development, design, and maintenance of our... ...challenge of building enterprise-grade software that millions of users depend on... ...design through implementation, testing, deployment, and post-launch...Full timeLocal areaWorldwideOkta
Toronto, ON6 days ago- ...creative minds, driven by a shared mission to engineer the future and safeguard our planet. As... ...built for tomorrow's challenges. The Test Engineer plays a pivotal role in... ...improvement of performance or to obtain data for development, standardization, or quality control....Full timeWork at office
Armstrong Fluid Technology
Toronto, ONa month ago $149k - $220k per year
...ecosystem. On the Commercial Software team, we are building the... .... We're a full stack web development team that works closely with... ...a team of multidisciplinary Engineers and Research Scientists using... ...processing, cloud infrastructures, test-driven and API-driven...Full timeInternshipManual laborWork at officeWork from homeFlexible hoursWaabi
Toronto, ON7 days ago- Lead technical innovation as a Principal Engineer at Intact, focused on InsurTech Finance Core Systems. Utilize your expertise in architecture... ...proof-of-concepts Requirements: • Senior-level experience in software engineering or related discipline • Strong hands-on experience...
$136k - $187k per year
...If you are too, let's talk. We are looking for a Senior Software engineer who is passionate about writing the tools to integrate and... ...fast-paced agile environment. You will share our passion for test-driven development, continuous integration and automation to produce frequent...Full timeLocal areaWorldwideFlexible hoursOkta
Toronto, ON14 days ago$90k - $175k per year
...domain, you will also: Custom development: Architect cloud-native... ...shaping modernization and product engineering solutions from pursuit... ...middleware solutions (e.g., MuleSoft, Software AG, IBM), leading API, event-... ...engineering propositions—LLM/SLM integration, RAG...Permanent employmentFlexible hours$74.41k - $93k per year
...for ensuring that products are tested and certified to industry and... ...R&D testing for the broader engineering teams, along with other... ...develop test plans to product development engineers based on past experience... .... ~ Solidworks CAD and PDM software (or other 3D CAD/PDM software...Full timeWork at officeDiscover Your Next Opportunity Below
Toronto, ONmore than 2 months ago- ...its Digital Operations. As the founding Software Engineer, Applied AI, you own that layer end-to-end... ...Code, Codex etc) as a core part of your development workflow — not just as shortcuts, but as... ...document: Stay ahead of the fast-moving LLM/AI landscape and document what works,...Full timeFlexible hours
Urban Ridge Supplies
Toronto, ON18 days ago $140.6k - $190.6k per year
...Manager, Software Engineering Content Tools • CoCounsel Agent Context Overview of the... ...Content Tools: design, implementation, testing, deployment, and ongoing operations. Equally... ...leader will navigate a distributed development model in which tools are built and contributed...Full timeContract workWork at officeLocal areaFlexible hours- ...currently have an opening on their team to add a Senior Software Engineer. The Senior Software Engineer will be... ...and interpret logs (Process/Atom Logs). Own development end to end from high-quality coding, unit testing, UAT support to Prod deployment. Hands-on experience...Full time
Hire DigITalent Inc.
Toronto, ON19 days ago $65k - $85k per year
...ROLE: SOFTWARE ENGINEER TEAM: MEDIA LOCATION: TORONTO (HYBRID) COMPANY OVERVIEW At... ...Platform Engineering, Security, and Software Development to create the cloud platform that... ...assessments, dependency management, and security testing. Support penetration testing,...Full timeSeasonal workSalt Xc
Toronto, ONa month ago
Do you want to receive more vacancies?
Subscribe and receive similar vacancies to Software Development Engineer in Test - GenAI / LLM. Be the first to apply!
Related searches