Senior Rag And Ai Evaluation Engineer
enableIT
Lead RAG troubleshooting and LLM evaluation with embeddings, vector search, retrieval optimization, and measurable quality testing.As a Senior Agentic AI Engineer, you will improve how enterprise AI applications retrieve information and generate grounded responses. The role combines RAG pipeline analysis, agentic workflow development, prompt engineering, and quality evaluation. You will work across ingestion, chunking, embeddings, retrieval, reranking, context, and generation.Key Responsibilities:
Troubleshoot RAG pipelines from ingestion through response generationOptimize retrieval quality, grounding, relevance, and response accuracyDesign evaluation strategies with metrics and test datasetsImplement prompt engineering, tool calling, and structured outputsSupport production troubleshooting and continuous application improvementRequirements:
Experience developing RAG solutions with embeddings and vector searchExperience evaluating LLM applications with regression testingHands-on agentic AI or LLM application development experienceStrong understanding of context management and multi-step workflowsExcellent communication of technical designs and engineering trade-offsCombine RAG optimization, agentic workflows, and rigorous evaluation to strengthen enterprise LLM application quality and grounding.
#J-18808-Ljbffr
Troubleshoot RAG pipelines from ingestion through response generationOptimize retrieval quality, grounding, relevance, and response accuracyDesign evaluation strategies with metrics and test datasetsImplement prompt engineering, tool calling, and structured outputsSupport production troubleshooting and continuous application improvementRequirements:
Experience developing RAG solutions with embeddings and vector searchExperience evaluating LLM applications with regression testingHands-on agentic AI or LLM application development experienceStrong understanding of context management and multi-step workflowsExcellent communication of technical designs and engineering trade-offsCombine RAG optimization, agentic workflows, and rigorous evaluation to strengthen enterprise LLM application quality and grounding.
#J-18808-Ljbffr
Vacancy posted 1 hour ago
Similar jobs that could be interesting for youBased on the Senior Rag And Ai Evaluation Engineer in Winnipeg, MB vacancy
- Fortiv Technical Inc. seeks an AI Engineer to build production-grade backend systems powering agentic AI workflows in a hybrid environment in Canada. You will design retrieval pipelines, evaluate LLM-powered applications, and collaborate with product and infrastructure teams...Senior
- ...Job Description Miratech is looking for a skilled Senior AI Engineer with Python, LangGraph and LangChain to join our US healthcare... ...applications, tool integrations, retrieval-augmented generation (RAG), and workflow automation. ~ Strong understanding of agentic...SeniorFull timeInternshipRemote workRelocation
- Join as a Senior AI Engineer driving innovative solutions through expertise in LLMs and Retrieval-Augmented Generation. Leverage your knowledge in... ..., with verified skills in deep learning, NLP, and advanced RAG pipeline implementation. You will focus on deploying GenAI models...Senior
$176k - $200k per year
We’re looking for a Senior Software Engineer to join our growing AI organization in Vancouver. Asana is pioneering a new category of technology as the human... ...across our core agentic runtime, develop high-throughput evaluation and safety infrastructure, or build the sophisticated...SeniorLong term contractWork at officeLocal areaWork from homeWorldwide- Design RAG systems and LLM features for Wiser Solutions remotely, using embeddings and retrieval to solve real customer problems.This Senior Machine Learning Engineer builds AI applications that reach production across product matching, classification, and extraction. You will...SeniorRemote work
$160k per year
Boston Consulting Group (BCG) in Toronto is looking for a Senior AI Tech Architect for their Platinion team. In this role, you will design... .... Ideal candidates should have a strong background in software engineering and experience with AI technologies. BCG offers a competitive...Senior- Infosys in Mississauga, Ontario, seeks a hands-on Gen AI / Agentic AI Engineer to design, develop, and deploy next‑generation AI solutions using LLMs, RAG, and agentic frameworks in a production‑oriented setting. You will collaborate with product and data teams, apply MLOps...Relocation
- Astra-North Infoteck Inc. is seeking an AI/ML engineer in Toronto to help develop Generative AI solutions. The role focuses on applying machine learning, deep learning, NLP, and RAG concepts to real business use cases, building reusable and efficient code in Python, and leveraging...
- ...Java-Holdings in Edmonton, Canada seeks an AI/ML Engineer to design, develop, integrate and deploy AI solutions that boost business processes... ...across the enterprise. The role emphasizes generative AI, LLMs, RAG and production AI operations with Microsoft technologies, third-...
- ...Drive AI innovation as a Senior AI Engineer in Mississauga, Canada. Craft Python-based AI applications using FastAPI or Flask while optimizing performance and scalability for enterprise solutions. We are seeking an experienced Senior AI Engineer with over 8 years in software...Senior
- Elevate AI advancement at DataAnnotation as an Electrical Engineer, focusing on real-world circuit and systems challenges. Apply your design and debugging skills... ...realistic engineering problems from practical experience • Evaluate AI model outputs against professional benchmarks •...
- Stripe is seeking an Engineering Manager for the AI Conversation Platform in Toronto. You will lead a growing ML-enabled team responsible for streaming feature pipelines, building ML models, and deploying them to production in a fast-paced, ambiguous environment. You will set...
- ...G2i Inc. is seeking a Senior Software Engineer, AI Training, to help evaluate the quality of AI-assisted coding interactions. This is a remote contract role with flexible hours and weekly commitment. You will review how models think like engineers, assess explanations,...SeniorContract workRemote workFlexible hours
- A technology company in Toronto is seeking an experienced software engineer to develop next-gen AI tools that enhance critical thinking. The ideal candidate will fine-tune language models and optimize prompts while collaborating with product and design teams. They should have...
- Advance RAG systems for insurance documents using embeddings, vector databases, and LLM... ...& King office.Ultrassure is seeking an AI Engineer to build intelligence for insurers, MGAs,... ...ingestion, prompt engineering, and model evaluation. You will collaborate with domain experts...Work at office
- A specialist AI consultancy in Canada is seeking a developer with extensive production Python experience to create innovative AI solutions in financial services. You will build key analytics systems and AI-driven pipelines, ensuring a tangible impact for major banks. The ideal...
- ...Senior Evaluation and Performance Analyst Department: Executive and Indigenous Affairs This Job Opening is limited to employees with the GNWT/WSCC in indeterminate and term positions. The Department of Executive and Indigenous Affairs works to provide overall management...SeniorLong term contract
- Evaluate frontier AI models with electrical engineering judgment, circuit analysis expertise, and flexible remote work from your own computer.DataAnnotation seeks an experienced electrical engineer for independent contract work evaluating real engineering tasks. Apply circuit...Contract workFor contractorsRemote workFlexible hours
- A leading technology firm in Canada is seeking a Senior AI Systems Architect to design the foundational intelligence layer for innovative AI-driven features. The role includes defining robust retrieval systems and ensuring cost control and reliability. Ideal candidates should...SeniorFull timeContract work10 hours per week
- Leverage your engineering expertise as an Electrical Engineer AI Evaluator with DataAnnotation, focusing on evaluating frontier AI models in real-world electronics tasks. Engage in challenging problem design and model assessment remotely. At DataAnnotation, we seek an experienced...Remote work
- Evaluate AI-generated code remotely for Alignerr while applying software engineering, debugging, and code review expertise. Work flexibly on an hourly contract from Calgary.... ...experienceApply software engineering, code evaluation, and precise technical communication to help...Hourly payContract workRemote work
- ...Coffee. Position Summary Join our AI team as an Intermediate AI Engineer, where you'll build and deploy AI... ...requirements into technical solutions. • Evaluate, monitor, and optimize models and... ...LLM Development, Prompt Engineering, RAG, AI Agent Frameworks (LangChain,...
- ...hireVouch is hiring a Staff Software Engineer, AI Engineering. This remote Canada role leads hands-on AI agent design across billing and revenue platforms, with Python and Java services, LangGraph, and LangChain at the core. You will drive production-grade AI services, ensure...Remote work
- Develop LLM and RAG applications with Tata Consultancy Services, using prompts, embeddings, and vector databases. Apply Responsible AI throughout delivery.As a Generative AI specialist, you will design, develop, and deploy AI applications supported by orchestration tools. Your...
- A leading AI solutions provider is seeking a Senior GenAI Engineer in Toronto, Ontario. This role focuses on designing and developing LLM-based solutions for an AI-powered recommendation platform. The ideal candidate will lead the architecting of scalable GenAI pipelines and...Senior
$140k - $175k per year
Job Description Thomson Reuters Labs is seeking a Lead Research Engineer to lead the design and delivery of production-ready software for artificial intelligence (AI) model evaluation and benchmarking. You will build scalable evaluation infrastructure that enables research...InternshipWork at officeLocal areaFlexible hours- Waabi, founded by AI visionary Raquel Urtasun, is the leader in Physical AI. With a world... ...positive way. To learn more visit: The Evaluation Algorithms team is responsible for... ...our highly realistic closed-loop simulation engine built with the latest in generative AI technologies...SeniorFull timeInternshipWork at officeWork from homeFlexible hours
- Apple’s Creativity Apps team seeks a Machine Learning Engineer focused on evaluating ML models and AI features in our creativity tools. You will define evaluation metrics, build evaluation datasets, and develop scalable infrastructure to ensure quality from model development...
- ...Rootly is hiring an AI Quality Engineer to own the evaluation and optimization of its agentic AI features. You will design test scenarios, probe how AI agents reason, and work with engineering to close the loop on performance across incident workflows. You will partner...
- ...support secure, cloud-native, AI-enabled applications for the financial... .... Using modern software engineering practices, AI-assisted... ...agentic workflows. In this senior role, you will be expected to... ...workflows, model-serving patterns, evaluation, data controls, fallback strategies...
Do you want to receive more vacancies?
Subscribe and receive similar vacancies to Senior Rag And Ai Evaluation Engineer. Be the first to apply!
Related searches
- senior financial accountant Winnipeg, MB
- senior sales consultant Winnipeg, MB
- senior communications specialist Winnipeg, MB
- senior bookkeeper Winnipeg, MB
- senior system administrator Winnipeg, MB
- senior technical recruiter Winnipeg, MB
- senior director Winnipeg, MB
- senior technical specialist Winnipeg, MB
- senior internal auditor Winnipeg, MB
- senior line cook Winnipeg, MB

