Lead Llm Inference Infrastructure Engineer
Jobgether
Build production inference infrastructure for large language models as a Canada-based Lead/Senior Engineer, optimizing latency, reliability, and costs.This hands-on engineering role develops scalable model-serving platforms for AI-driven professional applications. You will shape technical direction, engineering practices, and operational standards within a newly established team. Collaboration spans model training, evaluation, research, product, and engineering stakeholders.Key Responsibilities:
Build and operate production-grade model-serving infrastructure for large-scale AI workloadsOptimize batching, caching, quantization, latency, and operating costsStrengthen observability, monitoring, and operational visibilityIntegrate new model releases into serving infrastructureMentor engineers and contribute to technical strategyRequirements:
Hands-on Kubernetes or comparable infrastructure engineering experienceExperience operating model services through scaling and peak trafficLarge language model serving or inference optimization experienceStrong distributed systems scalability, availability, and reliability backgroundExperience troubleshooting production incidents under real trafficApply production infrastructure, distributed systems, and inference optimization expertise to advance reliable large language model serving.
#J-18808-Ljbffr
Build and operate production-grade model-serving infrastructure for large-scale AI workloadsOptimize batching, caching, quantization, latency, and operating costsStrengthen observability, monitoring, and operational visibilityIntegrate new model releases into serving infrastructureMentor engineers and contribute to technical strategyRequirements:
Hands-on Kubernetes or comparable infrastructure engineering experienceExperience operating model services through scaling and peak trafficLarge language model serving or inference optimization experienceStrong distributed systems scalability, availability, and reliability backgroundExperience troubleshooting production incidents under real trafficApply production infrastructure, distributed systems, and inference optimization expertise to advance reliable large language model serving.
#J-18808-Ljbffr
Vacancy posted 2 hours ago
Similar jobs that could be interesting for youBased on the Lead Llm Inference Infrastructure Engineer in Winnipeg, MB vacancy
$140k - $175k per year
...manages all applications and next steps. Our partner is looking for a Lead/Senior Inference Engineer based in Canada. The Lead/Senior Inference Engineer will build and operate the production inference infrastructure powering advanced large language models and AI-driven...SuggestedFlexible hours- FullStack is seeking a Senior AI Engineer to join our talent network and work with U.... ...and contribute to production-grade AI inference on GPU infrastructure. Requirements include 5+ years in AI/ML... ...stack experience (NIM, NeMo, TensorRT-LLM). Hybrid work with international...SuggestedFlexible hours
- Advance high-speed AI inference infrastructure at Cerebras Systems through latency optimization, reliability engineering, and production software.As a Staff Engineer, you will help build... ...across adjacent systems. Cerebras serves leading model labs, global enterprises, and AI-...Suggested
- ...Lead IT infrastructure projects at CBC/Radio-Canada using detailed engineering design, cloud technologies, and hands-on deployment expertise in a hybrid work model. This full-time permanent Engineering Project Lead position is based in Toronto, with projects focused on technology...SuggestedPermanent employmentFull time
- ...customers. Cohere is a team of researchers, engineers, designers, and more, who are passionate... ...experience running production infrastructure at a large scale ~ Experience designing... ...they influence latency and throughput of inference. ~ Strong understanding or working experience...SuggestedFull timeWork at officeRemote workFlexible hours
- Use LLM tools, JavaScript, and solution design to extend Orgvue’s organizational planning platform. This hybrid or remote role supports partners, customers, and internal teams.As Senior Solutions Engineer, you will help define, develop, and deliver innovative Orgvue capabilities...Remote work
- Cohere is a leading security-first enterprise AI company building foundation AI models and end-to-end products. We are hiring for a role that builds and owns pipelines for post-training data used to train large language models. You will collaborate with researchers and annotation...Remote workWorldwide
- ...production AI agents for Dutch using Python, LLM APIs, retrieval, and structured outputs... ...supports virtual veterinary care.As Lead AI/Data Engineer, you will develop AI services for pet... ...agents, rigorous evaluations, Python, and reliable LLM infrastructure. #J-18808-Ljbffr
- About FullStackFullStack is your AI-native engineering partner, built to turn AI capability... ...work as a Senior AI Engineer (NVIDIA + LLM Inference). You will integrate directly into... ...software engineering, or machine learning infrastructure. Advanced English is required....Full timeFlexible hours
$114.4k - $164.4k per year
Company:Qualcomm Canada ULC Job Area:Engineering Group, Engineering Group Machine Learning... ...Engineering General Summary:As a leading technology innovator, Qualcomm pushes... ...designed with machine learning software) for inference or training solutions. Develops...Immediate startWork from home- Lead LLM systems at Axya, working hands-on with Python, retrieval, agent workflows, and evaluation in a 100% remote environment.As Lead AI Engineer, you will own Axya’s AI architecture and technical direction. You will report to the CTO while partnering with Product, Customer...Remote work
- ...allows Cerebras to deliver industry-leading training and inference speeds; over 10 times faster than... ...fast decode on the Cerebras Wafer-Scale Engine. We are hiring a Software Engineer... ...stack, and rack-scale AMD GPU infrastructure, to make this new serving path reliable...
$180k - $360k per year
...Baseten powers mission-critical inference for the world's most dynamic... ...AI research, flexible infrastructure, and seamless developer tooling... ...and help build the platform engineers turn to ship AI products.About... ...runtime that powers large-scale LLM inference on Baseten. Our inference...Flexible hours$100k - $145k per year
Thomson Reuters is seeking a Senior Inference Engineer, AI. This person will collaborate with platform... ..., optimizing, and scaling AI and LLM workloads that power TR’s AI driven products... ...build the next generation of TR’s AI infrastructure, working alongside cloud engineering,...Work at officeLocal areaFlexible hours2 days per week3 days per week- ...allows Cerebras to deliver industry-leading training and inference speeds; over 10 times faster than... ...TTFT) at massive scale. As an Engineering Manager, Inference Router, you will... ...autoscaling, and the high‑availability infrastructure that keeps the platform fast and...
- Who are we? Cohere is the leading security-first enterprise AI company... ...is a team of researchers, engineers, designers, and more, who are... ...within our customers' custom infrastructure and data perimeters. In... ...ambiguous, customer-specific inference challenges—ranging from multi...Full timeWork at officeLocal areaRemote workHome office
- ...is on enabling more players to participate in creative expression by making creation easier, safer, and more rewarding.As Lead Infrastructure Engineer, you will own the GPU fleet our researchers train on including capacity, scheduling, diagnostics, and support. You will set...RemplacementFull timeWork at officeLocal area3 days per week
- ...ideas, courage and passion to drive life-changing impact to ZS.LLM Engineer We are seeking a highly skilled LLM Engineer with 1-4 years... ...responsible for designing and developing scalable software solutions, leading architecture design, and ensuring the performance and...Work at officeLocal areaWork from homeWorldwideFlexible hours
$180k - $360k per year
...Baseten powers mission-critical inference for the world's most dynamic... ...AI research, flexible infrastructure, and seamless developer tooling... ...and help build the platform engineers turn to ship AI products.About... ...contributions to the exciting field of LLM inference.EXAMPLE INITIATIVES...Flexible hours- Cohere is hiring a Backend Engineer to optimize model inference systems. You will build low-latency serving infrastructure, implement batching strategies, and develop the backend services... ...to Have Experience with vLLM, TensorRT-LLM, or similarFamiliarity with GPU memory...Work at office
$72.06k - $138.94k per year
At Capgemini Engineering, the world leader in engineering services, we bring... ...-grade large language model inference across current and next-... ...reproducible benchmarking against leading GPU platforms.Your roleImprove... ...management.Hands-on experience with an LLM serving framework such as vLLM...Full timeLocal area- ...GPUs. This architecture allows Cerebras to deliver industry-leading training and inference speeds; over 10 times faster than GPU-based hyperscale... ...About the Role We are seeking a versatile and experienced engineer to join our Inference Core Model Bringup team. This team...
$165k - $330k per year
...Baseten Baseten powers mission-critical inference for the world's most dynamic AI... ...By uniting applied AI research, flexible infrastructure, and seamless developer tooling, we enable... ...Capital. Join us and help build the platform engineers turn to ship AI products.THE ROLE As...InternshipFlexible hours- Enterprise infrastructure knowledge and delivery governance anchor this Lead opportunity with Interac. The role coordinates secure, resilient technology outcomes across... ...while influencing teams across Product, Engineering, Architecture, Security, Finance, Operations, and...
- Lead transportation infrastructure engineering in Calgary with AtkinsRéalis, applying road design expertise and team leadership to Alberta projects. Coordinate multidisciplinary delivery for major clients.As Transportation Infrastructure Team Lead, you will develop engineering...
- ...'s FX Technology team in Mississauga is seeking a Senior Infrastructure and Middleware Engineer to design, build, and support the application infrastructure... ...trading platforms. You will work with global teams, lead infrastructure design decisions, and drive high-performance...
- ...allows Cerebras to deliver industry-leading training and inference speeds; over 10 times faster than... ...fast decode on the Cerebras Wafer-Scale Engine. We are hiring a Software Engineer... ...enablement, compiler, runtime, cloud infrastructure, product, customer-facing teams, and...
- Shape Hootsuite’s cloud-based Linux infrastructure in Canada through infrastructure-as-code, reliability engineering, and collaborative technical leadership.This lead position guides SRE and DevOps practitioners while partnering across product development and department teams...Flexible hours
- Join OpenText as a Lead Infrastructure Engineer and leverage your extensive experience to enhance enterprise systems. This senior position focuses on Microsoft technologies, ensuring robust and scalable solutions. As part of the Corporate Technology Platform Engineering team...
- ...allows Cerebras to deliver industry-leading training and inference speeds; over 10 times faster than... ...About the RoleWe're hiring a Software Engineer to help contribute to projects on... ...Influence. Partner with ML, Product, Infrastructure, and Cloud teams to translate product...
Do you want to receive more vacancies?
Subscribe and receive similar vacancies to Lead Llm Inference Infrastructure Engineer. Be the first to apply!

