Software Engineer, AI Systems

This position is listed on behalf of a partner company, who manages all applications and next steps. Our partner is looking for a Software Engineer, AI Systems based in Canada. This is a hands-on AI engineering role focused on building and operating production-grade LLM systems for enterprise applications. You will design multi-step reasoning workflows that combine models, tools, retrieval, and knowledge graphs. The role sits at the intersection of AI engineering, backend development, evaluation, and production operations. You will help transform complex evidence into traceable, reliable outputs that support safety-critical decision-making. Working in a small, high-ownership environment, you will collaborate closely with technical and product leaders. You will have significant influence over architecture, model selection, evaluation practices, and the AI roadmap. The position offers the opportunity to work on technically challenging systems where precision, observability, and human oversight are essential. Accountabilities - Build, deploy, and operate production LLM pipelines that coordinate model calls, tools, graph queries, retrieval, quality gates, and specialized agent handoffs. - Extend agent orchestration workflows that move enterprise incidents from evidence gathering through analysis, review, and organizational learning. - Design grounding and retrieval systems using graph traversal, vector search, and hybrid retrieval to provide models with relevant evidence and organizational knowledge. - Develop robust evaluation frameworks, including representative datasets, scoring systems, regression suites, model comparisons, and human-review workflows. - Implement production AI operations capabilities covering tracing, tool-call auditing, cost and latency monitoring, failure handling, and quality dashboards.

- Identify and address hallucinations, workflow loops, silent quality degradation, and other production issues before they affect customers. - Evaluate and select AI models across major providers, balancing quality, latency, cost, context limitations, and operational risk. - Partner with product and knowledge engineering teams to shape technical direction, architecture, and the broader AI roadmap. - Contribute to secure enterprise AI practices, including tenant isolation, access controls, context protection, and safeguards against prompt injection. Requirements - Experience in AI or ML engineering with a track record of shipping production LLM systems used by real users. - Hands-on experience developing and debugging multi-step, tool-calling or agentic workflows using LangGraph, LangChain, or an equivalent framework. - Strong understanding of LLM evaluation, including representative datasets, regression testing, LLM-as-judge approaches, and/or human evaluation loops. - Practical experience designing retrieval and context-assembly strategies, with a clear understanding of what information should be retrieved, how much context is appropriate, and why. - Proven ownership of production systems through deployment, monitoring, incident response, and ongoing improvement, including experience diagnosing and resolving failures or regressions. - Ability to work across multiple model providers and make informed trade-offs involving quality, latency, cost, context, and operational risk. - Experience with Neo4j and Cypher or a comparable graph database, with the ability to understand and contribute to graph data modeling. - Strong Python skills and production experience with technologies such as FastAPI, asynchronous services, automated testing, observability, and maintainable software interfaces. - Experience with graph technologies such as Cypher, schema evolution, MERGE patterns, embeddings, or operating production knowledge graphs is an asset.