Sign up to access all features of our service.
  • Job search
  • Favorites
  • Create a CV
    New
  • Salaries
  • Subscriptions

Senior Rag And Ai Evaluation Engineer

enableIT

Lead RAG troubleshooting and LLM evaluation with embeddings, vector search, retrieval optimization, and measurable quality testing.As a Senior Agentic AI Engineer, you will improve how enterprise AI applications retrieve information and generate grounded responses. The role combines RAG pipeline analysis, agentic workflow development, prompt engineering, and quality evaluation. You will work across ingestion, chunking, embeddings, retrieval, reranking, context, and generation.Key Responsibilities:
Troubleshoot RAG pipelines from ingestion through response generationOptimize retrieval quality, grounding, relevance, and response accuracyDesign evaluation strategies with metrics and test datasetsImplement prompt engineering, tool calling, and structured outputsSupport production troubleshooting and continuous application improvementRequirements:
Experience developing RAG solutions with embeddings and vector searchExperience evaluating LLM applications with regression testingHands-on agentic AI or LLM application development experienceStrong understanding of context management and multi-step workflowsExcellent communication of technical designs and engineering trade-offsCombine RAG optimization, agentic workflows, and rigorous evaluation to strengthen enterprise LLM application quality and grounding.
#J-18808-Ljbffr
Vacancy posted 1 hour ago
Similar jobs that could be interesting for youBased on the Senior Rag And Ai Evaluation Engineer in Winnipeg, MB vacancy
  • Fortiv Technical Inc. seeks an AI Engineer to build production-grade backend systems powering agentic AI workflows in a hybrid environment in Canada. You will design retrieval pipelines, evaluate LLM-powered applications, and collaborate with product and infrastructure teams... 
    Senior

    Fortiv Technical Inc.

    Winnipeg, MB
    1 hour ago
  •  ...Job Description Miratech is looking for a skilled Senior AI Engineer with Python, LangGraph and LangChain to join our US healthcare...  ...applications, tool integrations, retrieval-augmented generation (RAG), and workflow automation. ~ Strong understanding of agentic... 
    Senior
    Full time
    Internship
    Remote work
    Relocation

    Miratech

    Winnipeg, MB
    9 days ago
  • Join as a Senior AI Engineer driving innovative solutions through expertise in LLMs and Retrieval-Augmented Generation. Leverage your knowledge in...  ..., with verified skills in deep learning, NLP, and advanced RAG pipeline implementation. You will focus on deploying GenAI models... 
    Senior

    Tekshapers

    Winnipeg, MB
    1 hour ago
  • $176k - $200k per year

    We’re looking for a Senior Software Engineer to join our growing AI organization in Vancouver. Asana is pioneering a new category of technology as the human...  ...across our core agentic runtime, develop high-throughput evaluation and safety infrastructure, or build the sophisticated... 
    Senior
    Long term contract
    Work at office
    Local area
    Work from home
    Worldwide

    ContentBuffer

    Winnipeg, MB
    1 hour ago
  • Design RAG systems and LLM features for Wiser Solutions remotely, using embeddings and retrieval to solve real customer problems.This Senior Machine Learning Engineer builds AI applications that reach production across product matching, classification, and extraction. You will... 
    Senior
    Remote work

    Wiser Solutions

    Winnipeg, MB
    1 hour ago
  • $160k per year

    Boston Consulting Group (BCG) in Toronto is looking for a Senior AI Tech Architect for their Platinion team. In this role, you will design...  .... Ideal candidates should have a strong background in software engineering and experience with AI technologies. BCG offers a competitive... 
    Senior

    Boston Consulting Group

    Winnipeg, MB
    1 hour ago
  • Infosys in Mississauga, Ontario, seeks a hands-on Gen AI / Agentic AI Engineer to design, develop, and deploy next‑generation AI solutions using LLMs, RAG, and agentic frameworks in a production‑oriented setting. You will collaborate with product and data teams, apply MLOps... 
    Relocation

    Infosys

    Winnipeg, MB
    1 hour ago
  • Astra-North Infoteck Inc. is seeking an AI/ML engineer in Toronto to help develop Generative AI solutions. The role focuses on applying machine learning, deep learning, NLP, and RAG concepts to real business use cases, building reusable and efficient code in Python, and leveraging... 

    Astra-North Infoteck Inc.

    Winnipeg, MB
    1 hour ago
  •  ...Java-Holdings in Edmonton, Canada seeks an AI/ML Engineer to design, develop, integrate and deploy AI solutions that boost business processes...  ...across the enterprise. The role emphasizes generative AI, LLMs, RAG and production AI operations with Microsoft technologies, third-... 

    Java Holdings

    Winnipeg, MB
    1 hour ago
  •  ...Drive AI innovation as a Senior AI Engineer in Mississauga, Canada. Craft Python-based AI applications using FastAPI or Flask while optimizing performance and scalability for enterprise solutions. We are seeking an experienced Senior AI Engineer with over 8 years in software... 
    Senior

    Talentola

    Winnipeg, MB
    10 days ago
  • Elevate AI advancement at DataAnnotation as an Electrical Engineer, focusing on real-world circuit and systems challenges. Apply your design and debugging skills...  ...realistic engineering problems from practical experience • Evaluate AI model outputs against professional benchmarks •... 

    DataAnnotation

    Winnipeg, MB
    1 hour ago
  • Stripe is seeking an Engineering Manager for the AI Conversation Platform in Toronto. You will lead a growing ML-enabled team responsible for streaming feature pipelines, building ML models, and deploying them to production in a fast-paced, ambiguous environment. You will set... 

    Stripe, Inc.

    Winnipeg, MB
    1 hour ago
  •  ...G2i Inc. is seeking a Senior Software Engineer, AI Training, to help evaluate the quality of AI-assisted coding interactions. This is a remote contract role with flexible hours and weekly commitment. You will review how models think like engineers, assess explanations,... 
    Senior
    Contract work
    Remote work
    Flexible hours

    G2i Inc.

    Winnipeg, MB
    1 hour ago
  • A technology company in Toronto is seeking an experienced software engineer to develop next-gen AI tools that enhance critical thinking. The ideal candidate will fine-tune language models and optimize prompts while collaborating with product and design teams. They should have... 

    Linkus Group

    Winnipeg, MB
    1 hour ago
  • Advance RAG systems for insurance documents using embeddings, vector databases, and LLM...  ...& King office.Ultrassure is seeking an AI Engineer to build intelligence for insurers, MGAs,...  ...ingestion, prompt engineering, and model evaluation. You will collaborate with domain experts... 
    Work at office

    Ultrassure Inc.

    Winnipeg, MB
    1 hour ago
  • A specialist AI consultancy in Canada is seeking a developer with extensive production Python experience to create innovative AI solutions in financial services. You will build key analytics systems and AI-driven pipelines, ensuring a tangible impact for major banks. The ideal... 

    Prospect 33

    Winnipeg, MB
    1 hour ago
  •  ...Senior Evaluation and Performance Analyst Department: Executive and Indigenous Affairs This Job Opening is limited to employees with the GNWT/WSCC in indeterminate and term positions. The Department of Executive and Indigenous Affairs works to provide overall management... 
    Senior
    Long term contract

    gov

    Winnipeg, MB
    10 days ago
  • Evaluate frontier AI models with electrical engineering judgment, circuit analysis expertise, and flexible remote work from your own computer.DataAnnotation seeks an experienced electrical engineer for independent contract work evaluating real engineering tasks. Apply circuit... 
    Contract work
    For contractors
    Remote work
    Flexible hours

    DataAnnotation

    Winnipeg, MB
    1 hour ago
  • A leading technology firm in Canada is seeking a Senior AI Systems Architect to design the foundational intelligence layer for innovative AI-driven features. The role includes defining robust retrieval systems and ensuring cost control and reliability. Ideal candidates should... 
    Senior
    Full time
    Contract work
    10 hours per week

    DarkRoast Design

    Winnipeg, MB
    1 hour ago
  • Leverage your engineering expertise as an Electrical Engineer AI Evaluator with DataAnnotation, focusing on evaluating frontier AI models in real-world electronics tasks. Engage in challenging problem design and model assessment remotely. At DataAnnotation, we seek an experienced... 
    Remote work

    DataAnnotation

    Winnipeg, MB
    1 hour ago
  • Evaluate AI-generated code remotely for Alignerr while applying software engineering, debugging, and code review expertise. Work flexibly on an hourly contract from Calgary....  ...experienceApply software engineering, code evaluation, and precise technical communication to help... 
    Hourly pay
    Contract work
    Remote work

    Alignerr Corp.

    Winnipeg, MB
    1 hour ago
  •  ...Coffee. Position Summary Join our AI team as an Intermediate AI Engineer, where you'll build and deploy AI...  ...requirements into technical solutions. • Evaluate, monitor, and optimize models and...  ...LLM Development, Prompt Engineering, RAG, AI Agent Frameworks (LangChain,... 

    Price Industries Limited

    Winnipeg, MB
    10 days ago
  •  ...hireVouch is hiring a Staff Software Engineer, AI Engineering. This remote Canada role leads hands-on AI agent design across billing and revenue platforms, with Python and Java services, LangGraph, and LangChain at the core. You will drive production-grade AI services, ensure... 
    Remote work

    hireVouch

    Winnipeg, MB
    1 hour ago
  • Develop LLM and RAG applications with Tata Consultancy Services, using prompts, embeddings, and vector databases. Apply Responsible AI throughout delivery.As a Generative AI specialist, you will design, develop, and deploy AI applications supported by orchestration tools. Your... 

    Tata Consultancy Services Limited

    Winnipeg, MB
    1 hour ago
  • A leading AI solutions provider is seeking a Senior GenAI Engineer in Toronto, Ontario. This role focuses on designing and developing LLM-based solutions for an AI-powered recommendation platform. The ideal candidate will lead the architecting of scalable GenAI pipelines and... 
    Senior

    Compunnel, Inc.

    Winnipeg, MB
    1 hour ago
  • $140k - $175k per year

    Job Description Thomson Reuters Labs is seeking a Lead Research Engineer to lead the design and delivery of production-ready software for artificial intelligence (AI) model evaluation and benchmarking. You will build scalable evaluation infrastructure that enables research... 
    Internship
    Work at office
    Local area
    Flexible hours

    Thomson Reuters

    Winnipeg, MB
    1 hour ago
  • Waabi, founded by AI visionary Raquel Urtasun, is the leader in Physical AI. With a world...  ...positive way. To learn more visit: The Evaluation Algorithms team is responsible for...  ...our highly realistic closed-loop simulation engine built with the latest in generative AI technologies... 
    Senior
    Full time
    Internship
    Work at office
    Work from home
    Flexible hours

    Waabi

    Winnipeg, MB
    1 hour ago
  • Apple’s Creativity Apps team seeks a Machine Learning Engineer focused on evaluating ML models and AI features in our creativity tools. You will define evaluation metrics, build evaluation datasets, and develop scalable infrastructure to ensure quality from model development... 

    Lex

    Winnipeg, MB
    1 hour ago
  •  ...Rootly is hiring an AI Quality Engineer to own the evaluation and optimization of its agentic AI features. You will design test scenarios, probe how AI agents reason, and work with engineering to close the loop on performance across incident workflows. You will partner... 

    Rootly

    Winnipeg, MB
    1 hour ago
  •  ...support secure, cloud-native, AI-enabled applications for the financial...  .... Using modern software engineering practices, AI-assisted...  ...agentic workflows. In this senior role, you will be expected to...  ...workflows, model-serving patterns, evaluation, data controls, fallback strategies... 

    The Citco Group Limited

    Winnipeg, MB
    1 hour ago

Do you want to receive more vacancies?

Subscribe and receive similar vacancies to Senior Rag And Ai Evaluation Engineer. Be the first to apply!