Lead/Senior Inference Engineer
$140k - $175k per yearJobgether
This position is listed on behalf of a partner company, which manages all applications and next steps. Our partner is looking for a Lead/Senior Inference Engineer based in Canada.The Lead/Senior Inference Engineer will build and operate the production inference infrastructure powering advanced large language models and AI-driven professional applications. This hands-on engineering role focuses on developing a reliable, scalable model-serving platform while optimizing performance, latency, and infrastructure costs. Working within a newly established engineering team, you will help shape its technical direction, engineering practices, and operational standards from the ground up. You will collaborate closely with model training, evaluation, research, and product teams to bring new model releases into production. The role offers significant autonomy and the opportunity to influence infrastructure strategy, reliability, and international platform expansion in a high-stakes, compliance-focused environment.
Accountabilities
Build, maintain, and operate production-grade model-serving infrastructure supporting large-scale AI workloads.Implement and optimize inference techniques, including batching, caching, quantization, and comparable approaches, to improve latency and reduce operating costs without compromising model quality.Strengthen platform observability, monitoring, and operational visibility as traffic volumes and deployment requirements increase.Develop the operational and compliance foundations required for production growth, including controls and evidence supporting the ISO 27001 compliance path.Establish cost-tracking and attribution mechanisms that support financial planning, as well as GPU-capacity contingency plans that help prevent service interruptions and unnecessary parallel infrastructure costs.Evaluate and document trade-offs between cost, latency, and reliability, communicating technical recommendations to engineering leadership and partner teams.Work directly with model training and evaluation teams to integrate new model releases into the serving infrastructure.Help establish engineering practices and tools, including on-call processes, incident reviews, onboarding, and operational documentation.Collaborate with research, product, and engineering stakeholders to ensure the serving platform evolves alongside model capabilities and product requirements.Contribute to the team's technical strategy and mentor other engineers as the function grows.Requirements:
Hands-on infrastructure engineering experience with Kubernetes or comparable technologies, including ownership of production systems beyond prototyping.Experience operating and maintaining model services across different workloads, including peak traffic, scaling, disaster recovery, and service reliability.Direct experience with model-serving infrastructure and/or inference optimization for large language models, at a scale appropriate to previous roles.Strong background in large-scale distributed systems, particularly production scalability, availability, and reliability.Proven ability to troubleshoot distributed systems and resolve production incidents under real traffic conditions.Demonstrated experience delivering software end to end, from technical design and implementation through deployment and ongoing operations.Ability to establish and uphold engineering best practices while adapting to evolving product and business requirements.Strong autonomy, technical judgment, and ownership, with the ability to operate as an early lead engineer and mentor colleagues as the team expands.Preferred qualifications:
Experience with LLM-serving technologies such as vLLM, TensorRT-LLM, Triton, SGLang, or similar frameworks.Exposure to GPU-cluster capacity planning and/or multi-region or multi-cloud deployment environments.Experience working in regulated or compliance-intensive environments, including ISO 27001, SOC 2, or comparable standards.Familiarity with systems programming languages such as Rust or Go.Knowledge of systems-level performance optimization, including rate limiting, request routing, and identifying bottlenecks across networking, hardware, and lower-level software.
Benefits:
Competitive compensation: CAD $140,000–$175,000 annual base salary for Ontario, Canada.Potential eligibility for an annual performance bonus based on individual and organizational results.Comprehensive benefits and wellbeing programs, including retirement savings, flexible vacation, and employee incentive programs.Flexible work arrangements, including the possibility of working from anywhere for up to eight weeks per year.Two company-wide mental health days annually and access to the Headspace app.Learning and career development opportunities, including skills development and tuition reimbursement.Resources supporting mental, physical, and financial wellbeing.Two paid volunteer days per year, along with opportunities to participate in community projects and environmental and social impact initiatives.Inclusive workplace culture focused on collaboration, professional growth, flexibility, and work-life balance.
#J-18808-Ljbffr
Accountabilities
Build, maintain, and operate production-grade model-serving infrastructure supporting large-scale AI workloads.Implement and optimize inference techniques, including batching, caching, quantization, and comparable approaches, to improve latency and reduce operating costs without compromising model quality.Strengthen platform observability, monitoring, and operational visibility as traffic volumes and deployment requirements increase.Develop the operational and compliance foundations required for production growth, including controls and evidence supporting the ISO 27001 compliance path.Establish cost-tracking and attribution mechanisms that support financial planning, as well as GPU-capacity contingency plans that help prevent service interruptions and unnecessary parallel infrastructure costs.Evaluate and document trade-offs between cost, latency, and reliability, communicating technical recommendations to engineering leadership and partner teams.Work directly with model training and evaluation teams to integrate new model releases into the serving infrastructure.Help establish engineering practices and tools, including on-call processes, incident reviews, onboarding, and operational documentation.Collaborate with research, product, and engineering stakeholders to ensure the serving platform evolves alongside model capabilities and product requirements.Contribute to the team's technical strategy and mentor other engineers as the function grows.Requirements:
Hands-on infrastructure engineering experience with Kubernetes or comparable technologies, including ownership of production systems beyond prototyping.Experience operating and maintaining model services across different workloads, including peak traffic, scaling, disaster recovery, and service reliability.Direct experience with model-serving infrastructure and/or inference optimization for large language models, at a scale appropriate to previous roles.Strong background in large-scale distributed systems, particularly production scalability, availability, and reliability.Proven ability to troubleshoot distributed systems and resolve production incidents under real traffic conditions.Demonstrated experience delivering software end to end, from technical design and implementation through deployment and ongoing operations.Ability to establish and uphold engineering best practices while adapting to evolving product and business requirements.Strong autonomy, technical judgment, and ownership, with the ability to operate as an early lead engineer and mentor colleagues as the team expands.Preferred qualifications:
Experience with LLM-serving technologies such as vLLM, TensorRT-LLM, Triton, SGLang, or similar frameworks.Exposure to GPU-cluster capacity planning and/or multi-region or multi-cloud deployment environments.Experience working in regulated or compliance-intensive environments, including ISO 27001, SOC 2, or comparable standards.Familiarity with systems programming languages such as Rust or Go.Knowledge of systems-level performance optimization, including rate limiting, request routing, and identifying bottlenecks across networking, hardware, and lower-level software.
Benefits:
Competitive compensation: CAD $140,000–$175,000 annual base salary for Ontario, Canada.Potential eligibility for an annual performance bonus based on individual and organizational results.Comprehensive benefits and wellbeing programs, including retirement savings, flexible vacation, and employee incentive programs.Flexible work arrangements, including the possibility of working from anywhere for up to eight weeks per year.Two company-wide mental health days annually and access to the Headspace app.Learning and career development opportunities, including skills development and tuition reimbursement.Resources supporting mental, physical, and financial wellbeing.Two paid volunteer days per year, along with opportunities to participate in community projects and environmental and social impact initiatives.Inclusive workplace culture focused on collaboration, professional growth, flexibility, and work-life balance.
#J-18808-Ljbffr
Vacancy posted 2 hours ago
Similar jobs that could be interesting for youBased on the Lead/Senior Inference Engineer in Winnipeg, MB vacancy
- Build production inference infrastructure for large language models as a Canada-based Lead/Senior Engineer, optimizing latency, reliability, and costs.This hands-on engineering role develops scalable model-serving platforms for AI-driven professional applications. You will...Senior
$100k - $145k per year
Thomson Reuters is seeking a Senior Inference Engineer, AI. This person will collaborate with platform teams to enhance capacity forecasting for AI... ...first approach ensures you have the tools and knowledge to grow, lead, and thrive in an AI-enabled future. Industry Competitive...SeniorWork at officeLocal areaFlexible hours2 days per week3 days per week- ...This architecture allows Cerebras to deliver industry-leading training and inference speeds; over 10 times faster than GPU-based hyperscale... ...high-speed inference. About The RoleWe’re hiring a Senior Frontend Engineer to own and scale critical parts of the Cerebras...SeniorLong term contractShift work
- Staff + Senior Software Engineer, InferenceAbout AnthropicAnthropic’s mission is to create reliable, interpretable, and steerable AI systems. We... ...together to build beneficial AI systems. About the roleOur Inference team is responsible for building and maintaining the...SeniorLong term contractWork at officeWorldwideVisa sponsorshipFlexible hours
- Senior AI Engineer, AI Inference Systems Join to apply for the Senior AI Engineer, AI Inference Systems role at Brix .Location : Palo Alto... ...of LLM optimization.Bachelor or Master degree from a leading academic institution.Tech Stack Front end: Python, Flutter...Senior
- Build machine learning inference APIs with Python and distributed systems expertise at Cerebras Systems, connecting GPU prefill with Cerebras decode.As a Senior Software Engineer, you will develop production software across inference APIs, model integration, and request-routing...Senior
$155k - $205k per year
A leading technology company in Toronto, Ontario is looking for an ASIC Verification Engineer. As a key member of the ASIC Verification team, you will be responsible for verifying ASIC designs using advanced methodologies. The role requires a Bachelors Degree in EE, CS, or...Senior- ...Lead nuclear HVAC design with P.Eng. accountability, complex heat-load calculations, and multidisciplinary execution from Mississauga, Ontario. This senior engineering role covers HVAC and mechanical systems for Nuclear Power Plants. You will lead detailed design, supervise...Senior
- A leading AI technology company is seeking a Member of Technical Staff to enhance model efficiency. This role involves improving performance metrics, optimizing bottlenecks, and collaborating with various teams. The ideal candidate has 5+ years in high-performance coding, strong...SeniorFlexible hours
- Cohere, a leading security-first enterprise AI company, is seeking a Staff Forward Deployed Engineer to architect and deploy high-performance AI models inside enterprise environments... ...You will own the architecture for complex inference challenges, enable low latency and high...Senior
- Enhance AI capabilities as a Senior Inference Engineer with Thomson Reuters, where hybrid work enables flexibility. Collaborate with cross-functional teams to deploy and scale cutting-edge AI workloads. In this influential role, you will be embedded within Platform Engineering...Senior
- This role is for a highly experienced Back-End Software Engineer who wants to work on core systems that shape how the platform (Model Vault... ...do not have off-the-shelf solutionsModel Vault is Cohere’s inference platform for deploying and operating models securely at enterprise...SeniorRemote work
$165k - $330k per year
About Baseten Baseten powers mission-critical inference for the world's most dynamic AI companies, like Cursor, Notion, OpenEvidence,... ...Partners, and Spark Capital. Join us and help build the platform engineers turn to to ship AI products.About Baseten Baseten powers...InternshipFlexible hours- AMD is seeking a Senior Software Development Engineer to own the end-to-end model execution stack on AMD Instinct... ...at scale and high-performance inference serving. You will ship LLMs on real hardware... ...run efficiently across generations. Lead multi-node training, develop...Senior
- ...This architecture allows Cerebras to deliver industry-leading training and inference speeds; over 10 times faster than GPU-based... ...with ultra-fast decode on the Cerebras Wafer-Scale Engine. We are hiring a Senior Software Engineer to build and evolve the ML API layer...Senior
- Who are we? Cohere is the leading security-first enterprise AI company... ...is a team of researchers, engineers, designers, and more, who are... ...ambiguous, customer-specific inference challenges—ranging from multi-... ...hardware validation. You will lead technical adoption end-to-end,...Full timeWork at officeLocal areaRemote workHome office
$111k - $160k per year
...realize their greatest potential.Title and Summary Senior Software Engineer (Sanctions Service & AI Inference)Who is Mastercard? Mastercard is a global... ...development, testing, deployment and production support.Lead technical discussions, perform code reviews and promote...SeniorInternshipWorldwide- A rapid-growth technology firm in Toronto is seeking a Staff LLMOps Engineer to lead the design and optimization of large language model infrastructure on the cloud. The ideal candidate has over 6 years of experience in DevOps and expertise in deploying LLMs in cloud environments...Senior
- ...architecture allows Cerebras to deliver industry-leading training and inference speeds; over 10 times faster than GPU-based... ...-token (TTFT) at massive scale. As an Engineering Manager, Inference Router, you will lead the team responsible for the intelligent control...
- ...GPUs. This architecture allows Cerebras to deliver industry-leading training and inference speeds; over 10 times faster than GPU-based hyperscale... ...prefill with ultra-fast decode on the Cerebras Wafer-Scale Engine. We are hiring a Software Engineer to productionize and...
- ...architecture allows Cerebras to deliver industry-leading training and inference speeds; over 10 times faster than... ...About the RoleWe're hiring a Software Engineer to help contribute to projects on our... ...judgment. Production Leadership. Lead on the hardest production issues and...
- ...GPUs. This architecture allows Cerebras to deliver industry-leading training and inference speeds; over 10 times faster than GPU-based hyperscale... ...our customers do. This is an excellent opportunity for engineers who enjoy infrastructure, automation, and debugging across...SeniorInternship
- NVIDIA Canada is seeking a seasoned ASIC Verification Engineer to verify inference accelerators. You will verify ASICs across unit, block and SoC levels using SystemVerilog, UVM, and automated verification flows within a global team. Requirements: a bachelor in EE/CS/CE and...Senior
- Advance high-speed AI inference infrastructure at Cerebras Systems through... ...optimization, reliability engineering, and production software.As a... ...demand. You will also guide senior engineers, make high-... ...adjacent systems. Cerebras serves leading model labs, global enterprises...Senior
- Lead Cohere's Inference as a Service business, combining AI infrastructure strategy with pricing and platform ownership. This globally dispersed company supports remote work culture.As a senior product leader, you will own a B2B infrastructure business end to end. You will...SeniorRemote work
- Cohere is hiring a Backend Engineer to optimize model inference systems. You will build low-latency serving infrastructure, implement batching strategies, and develop the backend services that power Cohere's enterprise API.Requirements 5+ years of backend engineering experienceStrong...Work at office
- Build AI inference systems with Python and PyTorch as a Backend Engineer, AI, powering production interactions through reliable, observable APIs.You will own the inference and orchestration layer between models and users. The work covers production backend systems, AI-powered...
- FullStack is seeking a Senior AI Engineer to join our talent network and work with U.S. clients on flexible, project-based development. You will... ...into client teams and contribute to production-grade AI inference on GPU infrastructure. Requirements include 5+ years in AI/ML...SeniorFlexible hours
- Cohere is looking for a Backend Engineer to specialize in optimizing model inference systems. In this remote-first role, you will create low-latency serving infrastructure for advanced backend services. Candidates for this position must have over five years of backend engineering...Remote work
$180k - $360k per year
...Baseten powers mission-critical inference for the world's most dynamic... ...and help build the platform engineers turn to ship AI products.... ...cutting-edge models with industry-leading performance, scalability, and... ...to LLM inference engines and frameworks such as vLLM,...Flexible hours
Do you want to receive more vacancies?
Subscribe and receive similar vacancies to Lead/Senior Inference Engineer. Be the first to apply!
Related searches
- lead structural engineer Winnipeg, MB
- lead process engineer Winnipeg, MB
- lead software engineer Winnipeg, MB
- senior financial accountant Winnipeg, MB
- senior sales consultant Winnipeg, MB
- senior communications specialist Winnipeg, MB
- senior bookkeeper Winnipeg, MB
- senior system administrator Winnipeg, MB
- senior technical recruiter Winnipeg, MB
- senior director Winnipeg, MB

