Ai Quality Engineer: Prompt Testing & Evaluation
Rootly
Rootly is hiring an AI Quality Engineer to own the evaluation and optimization of its agentic AI features. You will design test scenarios, probe how AI agents reason, and work with engineering to close the loop on performance across incident workflows.You will partner with product and engineering for new AI releases, define quality metrics, and stay current on evaluation techniques. This role offers a fast-paced, high-impact environment with growth opportunities.
#J-18808-Ljbffr
#J-18808-Ljbffr
Vacancy posted 2 hours ago
Similar jobs that could be interesting for youBased on the Ai Quality Engineer: Prompt Testing & Evaluation in Winnipeg, MB vacancy
- Take on a pivotal role as an AI Quality Assurance and Testing Engineer, focusing on the assessment of machine learning and generative AI models. Ensure high standards of quality, bias assessment, and responsible AI practices across applications. In this short-term contract...SuggestedTemporary work
Infotek Consulting Inc.
Winnipeg, MB2 hours ago - Apple’s Creativity Apps team seeks a Machine Learning Engineer focused on evaluating ML models and AI features in our creativity tools. You will define evaluation... ..., and develop scalable infrastructure to ensure quality from model development to customer delivery. You will...Suggested
Lex
Winnipeg, MB2 hours ago - Shape trusted AI evaluation with performance testing and CI/CD expertise as a Software Development Engineer in Test at CrowdStrike. Work on scalable cybersecurity services.As an SDET, you will help define how AI agents are tested, evaluated, and trusted in production. CrowdStrike...Suggested
CrowdStrike Canada ULC
Winnipeg, MB2 hours ago - ...building in our weekly changelog. Rootly is building the AI-native future of incident management, and we need someone... ...our AI to its limits before our customers do. As our AI Quality Engineer, you'll own the evaluation and optimization of Rootly's agentic AI features --...SuggestedInternshipHome office
Rootly
Winnipeg, MB2 hours ago - Build reliable autonomous AI at Equifax through LLM evaluation, Python automation, and multi-agent workflow testing. This newly created position focuses on production reliability.As an Agentic AI Evaluation & Tuning Engineer, you will connect LLM capabilities with dependable...Suggested
Equifax Inc.
Winnipeg, MB2 hours ago $95k - $145k per year
....Job Responsibilities:This role involves testing AI models and software for diverse use cases... ...robust testing framework to ensure high quality standards. Key tasks include engaging with... ..., the role supports the R&D project team engineers and researchers by providing technical evaluation...Local areaMagna International
Winnipeg, MB2 hours ago- GenAI Prompt EngineerJOB SUMMARY The developer will be working within... ...including ML and Gen AI. The candidate should be familiar... ...python development and prompt engineering. The candidate should be able... ...different LLM models, conduct testing and performance evaluation is...
Moonfire
Winnipeg, MB2 hours ago - ...intelligence artificielle. Titre du poste : Ingénieur Prompt (Prompt Engineer)Lieu : Montréal, QuébecResponsabilités principalesLe développeur... ...de grande taille (LLM), ainsi qu’avec les activités de tests et d’évaluation des performances, est requise. Le candidat idéal devra...Apprenticeship
Genpact Ltd.
Winnipeg, MB2 hours ago - Test GenAI and chatbot responses for accuracy, relevance, and consistency at Tata Consultancy Services. Use Jira, Confluence... ...and Agile delivery practices.This SDET position combines quality assurance with conversational AI validation. You will identify hallucinations, incorrect...
Tata Consultancy Services Limited
Winnipeg, MB2 hours ago - ...SKF at. Position Overview We are seeking a skilled Quality Electrical Engineer to support customer and manufacturing quality for electrical... .... Key Responsibilities Failure Analysis and Testing Test customer-returned and internally non-conforming products...Permanent employment
- ...commitment to people. Our dependable quality, reliable delivery, and... ...Coffee. Position Summary Join our AI team as an Intermediate AI Engineer, where you'll build and deploy AI applications... ...into technical solutions. • Evaluate, monitor, and optimize models and applications...
- ...Astek Canada recherche un Prompt Engineer en IA générative pour rejoindre une escouade ML œuvrant au sein d'une organisation internationale... ...besoins métier en prompts efficaces et en code Python robuste, puis évaluerez la performance des solutions sur le long terme. Vous...
Astek Group
Winnipeg, MB2 hours ago - Software quality and test automation define this remote-first Quality Engineer role at StackAdapt, open to candidates anywhere in Canada.StackAdapt seeks a Quality Engineer for a cross-functional agile environment supporting successful product releases. You will build reliable...Remote work
StackAdapt Inc.
Winnipeg, MB2 hours ago $140k - $175k per year
...Reuters Labs is seeking a Lead Research Engineer to lead the design and delivery of... ...software for artificial intelligence (AI) model evaluation and benchmarking. You will build scalable... ...large language models (LLMs), measure quality, cost, and latency, and make informed decisions...InternshipWork at officeLocal areaFlexible hoursThomson Reuters
Winnipeg, MB2 hours ago- Elevate AI advancement at DataAnnotation as an Electrical Engineer, focusing on real-world circuit and systems challenges. Apply your design and debugging skills... ...realistic engineering problems from practical experience • Evaluate AI model outputs against professional benchmarks •...
DataAnnotation
Winnipeg, MB2 hours ago - ...Magna International in Canada is seeking an experienced engineer to test AI models and supporting software for diverse automotive applications... ...annotated datasets, and a robust testing framework to ensure high quality results. You will collaborate with universities and...
Magna International
Winnipeg, MB2 hours ago - Knotch needs an AI-focused QA Manager to validate changing outputs... ...will lead hands-on software testing while managing automation... ...deterministic behavior, output quality, edge cases, performance, and... ...partner with agile product and engineering teams while mentoring distributed...Remote work
Knotch
Winnipeg, MB2 hours ago - ...(...Si vous l'acceptez !) :En tant que Prompt Engineer en IA générative, vous intégrerez une escouade... ...et en code Python robuste, puis évaluer et suivre la performance des solutions dans... ...nouvelles versions des LLMRéaliser des tests réguliers et des analyses de performanceParticiper...Daily paidApprenticeshipWork at officeDay shift
Astek Group
Winnipeg, MB2 hours ago - GenAI prompt engineering and Python development define this role within a machine learning team developing Artificial Intelligence... ...position requires OpenAI and large language model experience, testing, performance evaluation, and ongoing monitoring. You will contribute to shared...
Moonfire
Winnipeg, MB2 hours ago - Evaluate AI-generated mechanical designs and calculations with strong engineering fundamentals and technical writing skills. Work fully remotely on flexible, project-based assignments... ...work focuses on identifying technical errors, testing real-world applicability, and comparing...For contractorsRemote workFlexible hours
Brunel
Winnipeg, MB2 hours ago - ...there's no telling what you could accomplish. The Creativity Apps team is looking for a Machine Learning Engineer focused on the quality and evaluation of ML models and AI features in our creativity applications. You will define how we measure whether the product and AI...
Apple, Inc.
Winnipeg, MB2 hours ago - Evaluate generative AI systems as a Quality Engineer in Toronto, using Python and test automation within a hybrid, full-time role.Epsilon Solutions LTD seeks an AI evaluator and quality engineer for generative-AI solutions. The position defines evaluation approaches for models...Full time
Best Your Buy
Winnipeg, MB2 hours ago - Build and evaluate production-grade Rust services remotely from Canada, applying backend engineering expertise to AI training and software validation.This senior-level work centers on backend... ...functional solutions and reproducible tests while addressing APIs, data persistence,...Work at officeRemote work
Jobgether
Winnipeg, MB2 hours ago - Shape reliable AI systems through retrieval, RAG, evaluation, and monitoring for an energy platform supporting complex portfolio decisions... ...or Node.Js daily.As a mid-to-senior Applied AI Engineer, you will improve how AI quality, cost, and latency are measured in production....
Luxoft
Winnipeg, MB2 hours ago - Evaluate AI-generated code at Turing using Python, TypeScript, or another production language. Work... ...-world repositories and assess technical quality.This independent contractor role suits experienced, hands-on software engineers with five or more years of experience. You...For contractors
Turing Enterprises, Inc.
Winnipeg, MB2 hours ago - ...We are seeking a highly experienced and hands-on Supplier Quality Engineer (SQE) with strong expertise in casting and machining processes, particularly aluminum castings. The role requires close collaboration with suppliers, frequent onsite visits, and active involvement in...
- ...Workada seeks an AI Evaluation Specialist to assess outputs from AI models across text, images, video and documents, applying detailed criteria... .... You will follow precise guidelines, contribute to higher-quality systems, and collaborate with a mission-driven team focused on...Full timeRemote work
Brock University
Winnipeg, MB2 hours ago - Salesforce is seeking a software engineer to contribute across architecture, design, and delivery... ...frontend components. You will work with AI-first approaches, building secure,... ...Site Reliability teams. The role emphasizes prompt engineering, API design, and observability...
Salesforce Partner Canada
Winnipeg, MB2 hours ago - Lead AI-assisted testing and in-sprint automation as a hands-on Senior SDET supporting quality engineering across global teams.This Senior SDET drives practical test automation and performance engineering while providing global technical leadership. You will integrate AI-driven...Shift work
IT Connex
Winnipeg, MB2 hours ago - Senior Quality Assurance Specialist (AI Testing / QA Lead) Location: Toronto, ON | Hybrid, 3 days per week onsite, 2 days remote Language: English... ...applications for response accuracy, hallucinations, prompt injection, security, privacy, and safety You’ll design...Contract workRemote work3 days per weekWeekday work
Altis Technology
Winnipeg, MB2 hours ago
Do you want to receive more vacancies?
Subscribe and receive similar vacancies to Ai Quality Engineer: Prompt Testing & Evaluation. Be the first to apply!
Related searches
- quality engineer Winnipeg, MB
- ingénieur qualité Winnipeg, MB
- technicien qualité Winnipeg, MB
- quality engineer manufacturing Winnipeg, MB
- quality engineer- qms Winnipeg, MB
- entry level quality engineer Winnipeg, MB
- directeur amelioration continue audit qualite Winnipeg, MB
- stage qualité Winnipeg, MB
- qualité agroalimentaire Winnipeg, MB
- qualité santé Winnipeg, MB