Ai Observability Engineer — Build & Govern Production Llms (Hybrid)
BMO
BMO Financial Group is seeking a Hybrid Senior AI Observability Engineer in Canada to design, build, and operate systems that track and evaluate non-deterministic AI models and agent workflows. You will trace LLM calls, monitor token usage, detect drift, and run automated evaluations in production.You should have 7–10 years of Site Reliability Engineering experience, strong automation and observability skills, and experience with Python, SQL, and telemetry tools (Prometheus, Grafana, Datadog).
#J-18808-Ljbffr
#J-18808-Ljbffr
Vacancy posted 1 hour ago
Similar jobs that could be interesting for youBased on the Ai Observability Engineer — Build & Govern Production Llms (Hybrid) in Winnipeg, MB vacancy
- WiseMatic Inc. is seeking a Senior AI Developer to lead the design and delivery of... ...for clients. You will architect production AI systems, build ML models, and ensure responsible AI practices. The role blends software engineering with practical AI judgement and mentorship...SuggestedWork at office
- Thomson Reuters is seeking a Senior Legal Engineer to design, build, and deploy AI-powered software for legal workflows. You will work with legal customers... ..., mentor engineers, and collaborate across engineering, product, and legal partners to improve the software lifecycle....Suggested
- ...for an Apps Development role focusing on GenAI technologies. The ideal candidate has 8-10 years of experience and strong expertise in LLMs, machine learning, and data science. You'll be responsible for deploying LLM-based applications and developing robust solutions using various...Suggested
- Nesto Group in Canada is seeking a senior software engineer to lead AI-enabled features for Maestro, our AI underwriting and lending engine... ...-based workflows, and optimize pipelines using Python and LLMs in production. The role emphasizes mentoring, architecture decisions, and...SuggestedRemote work
- Fortiv Technical Inc. seeks an AI Engineer to build production-grade backend systems powering agentic AI workflows in a hybrid environment in Canada. You will design retrieval pipelines, evaluate LLM-powered applications, and collaborate with product and infrastructure teams...Suggested
- A specialist AI consultancy in Canada is seeking a developer with extensive production Python experience to create innovative AI solutions... ...financial services. You will build key analytics systems and AI-driven... ...has a strong background in LLMs, deployed AI architectures, and...
$80k - $175k per year
...Family Group:Data Analytics & Reporting Hybrid role Designs, builds, and maintains systems that track, evaluate, and debug non-deterministic AI models and agent workflows. Trace LLM... ...drift, set up automated evaluations, and govern autonomous agent behaviors in production...Part time- ...Intercast Staffing Ltd. is seeking a Senior AI / Generative AI Engineer to design and deliver enterprise-grade AI... ...growing Data & AI Marketplace. You will build, scale, and operationalize advanced AI systems in secure, governed Azure environments. The role focuses on delivering...
- ...Altis in Toronto is seeking an Enterprise AI Engineer to lead AI adoption, development, and governance. The role collaborates with business leaders and IT across... ...organization to deliver responsible AI solutions that raise productivity and innovation. You’ll design and deploy AI...Remote work
- GEI is seeking an AI Engineer to develop AI solutions using pretrained models, Copilot technologies, and Azure-based tooling. You will build, deploy, and integrate AI capabilities into business workflows with a focus on security, scalability, and maintainability. You will collaborate...
$165k - $330k per year
...inference for the world's most dynamic AI companies, like Cursor, Notion,... ...AI to bring cutting-edge models into production. We're growing quickly and recently raised... ..., and Spark Capital. Join us and help build the platform engineers turn to ship AI products.About Baseten...Flexible hours- Join Honeycomb as a Software Engineer I, focusing on AI observability in a fully remote role. Contribute your software engineering expertise to enhance observability... ...engineering role, your responsibilities will include building observability tools and contributing to data models that...Remote work
- ...Stingray Group in Montreal is seeking an AI Software Developer to design, build, and deploy AI applications and agentic systems. You will contribute to our agent platform, working with product and engineering teams to translate business needs into practical AI solutions....
- ...will join a world-class team of engineers and data scientists from... ...ranging from leading startups building impactful technologiesto Fortune... ...building cutting-edge systems with LLMs and real-world data. In this... ...to design, prototype, and productize scalable machine learning solutions...Contract workRemote work
- Okta seeks an Observability Engineer for the Auth0 Platform Observability team. You will monitor, observe, and improve the platform’s reliability, partnering with Product and Platform Engineers to ship software rapidly while strengthening instrumentation and telemetry across...
- ...RBC is seeking a Lead AI Engineer to design, develop, and implement AI-driven solutions within the software development lifecycle. You will... ...to implement AI-powered automation, LLM-driven workflows, and governance frameworks, ensuring secure, scalable, and maintainable systems...
- Infosys in Mississauga, Ontario, seeks a hands-on Gen AI / Agentic AI Engineer to design, develop, and deploy next‑generation AI solutions using LLMs, RAG, and agentic frameworks in a production‑oriented setting. You will collaborate with product and data teams, apply MLOps...Relocation
- Elevate your impact as an AI Engineer focused on Generative AI technologies and data science. Leverage... ...RAG pipelines. Key Responsibilities: • Build LLM applications and tune models for... ...technology roles • Hands-on with various LLMs including Google Gemini and OpenAI models...
- Extend Tempo for AI and agent workloads through structured APIs, observability systems, and scalable engineering. Work remotely with modern coding assistants.As a Senior Engineer on Tempo, you will build capabilities for Grafana’s AI assistant, external integrators, and agent...Remote work
- ...Ample Insight Inc. is seeking a Data Scientist to join a world-class team of engineers and data scientists in Toronto. You will work on real-world problems, building production-grade AI/ML systems and turning cutting-edge research into working prototypes. The role emphasizes...
- ...leading manufacturer and distributor of air distribution products ( The founding principles of Price have never... ...free Tea and Coffee. Position Summary Join our AI team as an Intermediate AI Engineer, where you'll build and deploy AI applications, agents, and automation...
- BlueCat, Toronto, seeks a Product Manager to advance LiveNX and Network Synthetics across observability and DDI. You’ll partner with customers, engineering, design, and go‑to‑market teams to turn complex... ...product management, a passion for AI-driven outcomes, and strong...
- ...Closetech AI is seeking an AI Developer to design and build AI systems for client engagements and internal products, including PointO and Closetech EcoCore. You will balance rapid prototyping with production hardening to deliver reliable, scalable solutions. You’ll work...
- Join as a Senior AI Engineer driving innovative solutions through expertise in LLMs and Retrieval-Augmented Generation. Leverage your knowledge in AI fundamentals to build scalable applications. The role requires 8-10 years of relevant experience in Apps Development or systems...
- ...Java-Holdings in Edmonton, Canada seeks an AI/ML Engineer to design, develop, integrate and deploy AI solutions that boost business... ...across the enterprise. The role emphasizes generative AI, LLMs, RAG and production AI operations with Microsoft technologies, third-party platforms...
- NovoEd, Inc. is seeking a senior AI Engineer to design, build, and deploy production‑grade AI‑powered backend systems. The role emphasizes scalable LLM integration, vector database optimization, and high‑performance Python services. You will work across backend, AI inference...
- Solstice is building a Toronto-based engineering team to own core parts of our restaurant-grade pharma marketing platform. You will design APIs, background... ..., and deployment pipelines, shipping LLM agents to production while collaborating closely with co-founders Yiwen and...
- ...Lead responsible AI adoption at YVR through digital governance, IT service management, and practical controls supporting safe airport operations. As Manager, Technology Services, you will oversee AI and digital tool governance across YVR. Reporting to the Director, Data...
- A technology consulting firm in Canada seeks a Data Scientist with Machine Learning Engineer skills. This role involves exploring datasets, developing predictive models, and deploying machine learning models using frameworks like TensorFlow and PyTorch. Candidates should have...Full time
- Production troubleshooting and observability define this experienced Site Reliability Engineer role with Porter in Toronto’s Downtown Office.You will improve reliability, scalability, and performance through code, monitoring, and operational tooling. The role partners with developers...Work at office
Do you want to receive more vacancies?
Subscribe and receive similar vacancies to Ai Observability Engineer — Build & Govern Production Llms (Hybrid). Be the first to apply!

