Sign up to access all features of our service.
  • Job search
  • Favorites
  • Create a CV
    New
  • Salaries
  • Subscriptions

Ai Quality Engineer: Prompt Testing & Evaluation

Rootly

Rootly is hiring an AI Quality Engineer to own the evaluation and optimization of its agentic AI features. You will design test scenarios, probe how AI agents reason, and work with engineering to close the loop on performance across incident workflows.You will partner with product and engineering for new AI releases, define quality metrics, and stay current on evaluation techniques. This role offers a fast-paced, high-impact environment with growth opportunities.
#J-18808-Ljbffr
Vacancy posted 1 hour ago
Similar jobs that could be interesting for youBased on the Ai Quality Engineer: Prompt Testing & Evaluation in Winnipeg, MB vacancy
  •  ...SKF at. Position Overview We are seeking a skilled Quality Electrical Engineer to support customer and manufacturing quality for electrical...  .... Key Responsibilities Failure Analysis and Testing Test customer-returned and internally non-conforming products... 
    Suggested
    Permanent employment

    SKF Group

    Winnipeg, MB
    7 days ago
  •  ...commitment to people. Our dependable quality, reliable delivery, and...  ...Coffee. Position Summary Join our AI team as an Intermediate AI Engineer, where you'll build and deploy AI applications...  ...into technical solutions. • Evaluate, monitor, and optimize models and applications... 
    Suggested

    Price Industries Limited

    Winnipeg, MB
    7 days ago
  •  ...We are seeking a highly experienced and hands-on Supplier Quality Engineer (SQE) with strong expertise in casting and machining processes, particularly aluminum castings. The role requires close collaboration with suppliers, frequent onsite visits, and active involvement in... 
    Suggested

    Global Connect Technologies

    Winnipeg, MB
    7 days ago
  •  ...doors to new opportunities and growth.  THE POSITION   The  Quality Engineer  leads the development, implementation, and continuous...  ...through root cause analysis and implement corrective actions promptly.  Collaborate with suppliers to improve quality and drive continuous... 
    Suggested
    Full time
    Remote work

    Smartrend Manufacturing Group

    Winnipeg, MB
    16 days ago
  • Shape trusted AI evaluation at CrowdStrike using agent frameworks, LLM APIs, and verification-first test automation. Work on cloud-based cybersecurity systems.As a Software Development Engineer in Test, you will help determine how CrowdStrike AI agents are tested, evaluated... 
    Suggested

    CrowdStrike Inc.

    Winnipeg, MB
    1 hour ago
  •  ...Miratech is looking for a skilled Senior AI Engineer with Python, LangGraph and LangChain to join...  ...observability, monitoring, telemetry, and evaluation platforms. ~ Strong knowledge of software engineering best practices, testing, debugging, and performance optimization.... 
    Full time
    Internship
    Remote work
    Relocation

    Miratech

    Winnipeg, MB
    6 days ago
  •  ...Reporting to the Sr. Director, Quality, the Director, Quality Assurance leads...  ...products, and finished products; review of test results; evaluation of batch and production records; approval...  ...’s degree in Science, Pharmacy, Engineering, or a related field. An advanced degree... 
    Flexible hours

    Emergent Biosolutions

    Winnipeg, MB
    6 days ago
  • Take on a pivotal role as an AI Quality Assurance and Testing Engineer, focusing on the assessment of machine learning and generative AI models. Ensure high standards of quality, bias assessment, and responsible AI practices across applications. In this short-term contract... 
    Temporary work

    Infotek Consulting Inc.

    Winnipeg, MB
    1 hour ago
  • $85k - $225k per year

     ...Toronto is seeking an experienced professional to validate and ensure the reliability of Veeva AI Agents. The ideal candidate should have expertise in data quality, prompt engineering, and proficiency in Python. This full-time role offers a competitive salary range of $85,000... 
    Full time
    Remote work
    Flexible hours

    Veeva Systems

    Winnipeg, MB
    1 hour ago
  •  ...Join Workada as a Remote Video Production Evaluator dedicated to improving AI technology through precise evaluations. This full-time role focuses on quality assessment of multimedia outputs. At Workada, we seek an expert who can critically analyze video, image, and text... 
    Full time
    Remote work

    workada

    Winnipeg, MB
    7 days ago
  • Apple’s Creativity Apps team seeks a Machine Learning Engineer focused on evaluating ML models and AI features in our creativity tools. You will define evaluation...  ..., and develop scalable infrastructure to ensure quality from model development to customer delivery. You will... 

    Lex

    Winnipeg, MB
    1 hour ago
  •  ...Drive AI innovation as a Senior AI Engineer in Mississauga, Canada. Craft Python-based AI applications using FastAPI or Flask while optimizing performance and scalability for enterprise solutions. We are seeking an experienced Senior AI Engineer with over 8 years in software... 

    Talentola

    Winnipeg, MB
    7 days ago
  •  ...successful career in materials testing and inspection services by...  ...that’s rock solid. We perform quality assurance and quality control...  ...Preparation of field and laboratory test reports The successful...  ...experienced technologists and engineers. Your Capabilities and Credentials... 
    For contractors
    Flexible hours
    Weekend work

    Stantec

    Winnipeg, MB
    7 days ago
  •  ...ensure flight and product efficiency. As a Quality Inspector , you will ensure that all aircraft components, engines, and systems are maintained, repaired, or overhauled...  ...prior to the performance of any functional tests performed to verify safe completion of the product... 
    Long term contract
    Temporary work
    Traineeship
    Internship
    Immediate start
    Flexible hours
    Shift work
    Weekend work

    StandardAero

    Winnipeg, MB
    12 days ago
  •  ...protecting and saving lives. Job Summary  The Specialist I, Quality Compliance is responsible for supporting and representing...  ...audit observations, and author clear and concise audit reports. • Evaluate documentation, records, processes, and personnel practices to assess... 
    Flexible hours

    Emergent Biosolutions

    Winnipeg, MB
    6 days ago
  • Shape trusted AI evaluation with performance testing and CI/CD expertise as a Software Development Engineer in Test at CrowdStrike. Work on scalable cybersecurity services.As an SDET, you will help define how AI agents are tested, evaluated, and trusted in production. CrowdStrike... 

    CrowdStrike Canada ULC

    Winnipeg, MB
    1 hour ago
  • Quality Engineer Job Summary Reporting directly to the Quality Assurance Manager, Quality Engineer performs execution of Quality Engineering systems including; handling of nonconforming materials, Corrective/Preventive Actions, failure reporting, analysis and corrective action... 
    Full time

    Linamar Corporation

    Winnipeg, MB
    15 days ago
  • $118k - $152k per year

     ...experienced Software Developer with a strong understanding of AI-assisted development and prompt engineering. Ideally this person will have hands-on development...  .... Familiarity with prompt design, optimization, and testing techniques. Understanding of how AI technologies can... 
    Internship
    Immediate start

    Capco Group

    Winnipeg, MB
    1 hour ago
  •  ...largest Indigenous-owned software testing and technology services company. For...  ...who are passionate about quality, teamwork, and making an impact....  ...who are passionate about modern engineering practices and excited about how AI is changing the way software is tested... 
    Internship

    PLATO

    Winnipeg, MB
    1 hour ago
  • Become a key part of our team as an AI Quality Engineer, focusing on automated test development and quality assurance. This role combines Python expertise with a deep understanding of AI processes. With a minimum of 4 years in QA or test automation, you'll build and maintain... 

    Centraprise

    Winnipeg, MB
    1 hour ago
  •  ...The Quality Control Inspector will provide Quality Process Audit support to the Assembly Line. This position will assist in reducing process...  ...root cause analysis of quality issues. Verify production testing to ensure requirements and standards are met. WHAT YOU NEED... 
    Full time
    Temporary work

    Motor Coach Industries

    Winnipeg, MB
    11 days ago
  • $100k - $140k per year

     ...value expertise, curiosity, and continuous improvement — and we give you the space to grow. About the RoleWe are seeking AI Developer /Prompt Engineer to contribute to major projects. The AI Developer lead the design, development, and deployment of advanced AI-enabled... 
    Permanent employment
    Full time
    Remote work
    Flexible hours

    Levio Inc.

    Winnipeg, MB
    1 hour ago
  • $120k - $150k per year

     ...We’re also serious about technology. Make an impact as a Quality Engineering & Testing Manager to lead and strengthen quality engineering across the...  ...practical testing standards, advance automation and AI-enabled testing, promote test-driven development and other shift... 
    Work at office
    Remote work
    Flexible hours
    Shift work

    MNP

    Winnipeg, MB
    1 hour ago
  • Shape CoCounsel Legal with prompt engineering and legal expertise at Thomson Reuters in a hybrid work model. Evaluate artificial intelligence outputs trusted by legal professionals.As a Senior Legal AI Prompter and Evaluator, you will write prompts, skills, and agent configurations... 

    Thomson Reuters

    Winnipeg, MB
    1 hour ago
  •  ...Senior Evaluation and Performance Analyst Department: Executive and Indigenous Affairs This Job Opening is limited to employees with the GNWT/WSCC in indeterminate and term positions. The Department of Executive and Indigenous Affairs works to provide overall management... 
    Long term contract

    gov

    Winnipeg, MB
    7 days ago
  • Build reliable autonomous AI at Equifax through LLM evaluation, Python automation, and multi-agent workflow testing. This newly created position focuses on production reliability.As an Agentic AI Evaluation & Tuning Engineer, you will connect LLM capabilities with dependable... 

    Equifax Inc.

    Winnipeg, MB
    1 hour ago
  •  ...service, innovation and a commitment to people. Our dependable quality, reliable delivery, and knowledgeable follow-up service have made...  ...problem-solving skills. • The ability to read and interpret engineering drawings. • High proficiency in Microsoft office including,... 
    Permanent employment
    Full time
    Work at office

    Price Industries Limited

    Winnipeg, MB
    7 days ago
  •  ...bowl, or grabbing some groceries, our tech platform connects tens of millions of customers with hundreds of thousands of restaurant, grocery and convenience partners across the globe.About this role As a Senior Quality Engineer, you will work with an elite team of develop...
    Full time
    Night shift

    Just Eat Takeaway.com

    Winnipeg, MB
    15 days ago
  •  ...Director of Quality Assurance Winnipeg, MB Full-Time, Onsite POSITION SUMMARY: The Director of Quality Assurance - Transit...  ...from recurring. Partner closely with Manufacturing, Engineering, Supply Chain, Service, and Operations teams to solve complex challenges... 
    Full time
    Internship
    Immediate start

    New Flyer

    Winnipeg, MB
    20 days ago
  •  ...building in our weekly changelog. Rootly is building the AI-native future of incident management, and we need someone...  ...our AI to its limits before our customers do. As our AI Quality Engineer, you'll own the evaluation and optimization of Rootly's agentic AI features --... 
    Internship
    Home office

    Rootly

    Winnipeg, MB
    1 hour ago

Do you want to receive more vacancies?

Subscribe and receive similar vacancies to Ai Quality Engineer: Prompt Testing & Evaluation. Be the first to apply!