Sign up to access all features of our service.
  • Job search
  • Favorites
  • Create a CV
    New
  • Salaries
  • Subscriptions

Engineering Manager, Inference Router

Cerebras

Cerebras Systems builds the world's largest AI chip, 56 times larger than GPUs. This architecture allows Cerebras to deliver industry-leading training and inference speeds;
over 10 times faster than GPU-based hyperscale cloud inference services.
This order of magnitude increase in speed is transforming the user experience of AI applications, unlocking real-time iteration and increasing intelligence via additional agentic computation.
Cerebras works with the leading model labs, global enterprises, and cutting-edge AI-native startups. OpenAI recently announced a multi-year partnership with Cerebras, to deploy 750 megawatts of scale, transforming key workloads with ultra high-speed inference.
About the RoleThe Inference Router team at Cerebras builds the systems that decide where every inference request runs. We route traffic across all of Cerebras’ data centers and third-party integrations, balancing heterogeneous compute, maximizing cache utilization, and delivering the industry’s lowest time-to-first-token (TTFT) at massive scale.
As an
Engineering Manager, Inference Router, you will lead the team responsible for the intelligent control plane of the fastest generative AI inference platform in the world. Your team owns global request routing, prefill/decode (P/D) disaggregation, prefix‑aware routing for high cache hit rates, autoscaling, and the high‑availability infrastructure that keeps the platform fast and reliable under real‑time, bursty load.
This role operates at the intersection of distributed systems, Kubernetes‑based infrastructure, routing‑algorithm design, and machine learning. It combines technical leadership, people management, and execution ownership, with direct impact on the latency, throughput, and reliability experienced by every Cerebras inference customer.
What You’ll DoTechnical Leadership
Own the architecture and evolution of the inference routing layer across all Cerebras data centers and third-party inference integrations.
Guide the design of:
global, multi‑region request routing and load balancing;
routing for heterogeneous compute, including prefill/decode (P/D) disaggregated serving;
prefix‑aware /cache‑aware routing to maximize KV‑cache reuse and utilization;
low‑TTFT, high‑throughput request scheduling under concurrent, bursty workloads.
Drive the routing algorithms that trade off cache locality, queueing, load, and latency in real time.
Invest in simulation: build high‑fidelity models of the fleet and its workloads so the team can validate routing policies, failover behavior, and scaling decisions under a wide range of conditions without tying up precious accelerator capacity.
Partner with inference runtime, cloud/platform, hardware, and ML teams to optimize end‑to‑end performance.
Team Leadership
Build, manage, and grow a team of distributed systems, infrastructure, and ML systems engineers.
Provide technical direction, mentorship, and career development.
Foster a culture of ownership, velocity, and operational excellence.
Recruit top talent in distributed systems, Kubernetes/infrastructure, and routing/ML systems.
Execution & Delivery
Drive execution of complex, cross‑functional initiatives spanning inference runtime, cloud/infrastructure, and ML teams.
Own delivery of capabilities such as:
prefix‑ and cache‑aware routing policies;
P/D‑disaggregation‑aware scheduling and placement;
autoscaling and capacity management across heterogeneous fleets;
high availability, failover, and graceful degradation;
simulation andload‑generation frameworks that exercise the router under diverse traffic patterns, failure modes, and fleet topologies — without consuming scarce production compute;
observability,SLOs, and reliability across the routing stack.
Ensure high‑quality releases through strong testing, load/chaos validation, and operational rigor.
Platform & Performance Ownership
Scale the routing layer to handle very large volumes of concurrent requests at the lowest possible latency.
Drive improvements in TTFT, throughput, cache‑hit rate, and compute efficiency across the fleet.
Design for reliability and high availability: failover, redundancy, scale‑out, and fail‑closed behavior under partial outages.
Identify and prioritize technical debt and system bottlenecks.
Maintain Cerebras’ industry‑leading inference speed advantage.
Cross‑Functional Collaboration
Partner with:
inference runtime teams (execution and serving);
cloud/platformand SRE teams (deployment, scaling, reliability);
ML researchersand engineers (model enablement, routing signals);
third‑party integration partners.
Act as a bridge between infrastructure, routing, and production systems.
What You Bring
8+ years of experience in large‑scale software engineering, distributed systems, or infrastructure.
2+ years of engineering management experience.
Strong programming skills in
C++ and/or Go
for performance‑critical, concurrent production systems.
Deep experience operating production services on
Kubernetes : deployments, autoscaling, rollout strategies, and multi‑cluster/multi‑region topologies.
Proven track record building and operating
highly available, scalable
systems: load balancing, failover, scale‑out, and reliability engineering.
Experience designing or operating request routing, load balancing, scheduling, or traffic‑management systems at scale.
Experience with cloud infrastructure and best practices for building scalable microservices.
Preferred Skills
Experience with large-scale
LLM/inference serving
and the systems around it (e.G. vLLM, TensorRT‑LLM, SGLang, or comparable).
Familiarity with prefill/decode (P/D) disaggregation, KV‑cache management, and prefix/cache‑aware routing.
ML background: understanding of inference workloads, batching, and the performance characteristics of serving LLMs and multimodal models.
Experience with autoscaling and capacity management across heterogeneous compute.
Experience building simulation, modeling, or load‑testing frameworks for distributed systems to validate behavior cost‑effectively at scale.
Strong grounding in observability (metrics, tracing, SLOs) and incident response for high‑traffic production systems.
Why This Role MattersThe Inference Router is the control plane behind Cerebras’ core promise: the fastest AI inference in the world. The routing decisions your team makes — across data centers, across heterogeneous compute, with cache‑awareness and high availability — directly determine the latency, throughput, and reliability every customer experiences. Your work keeps real‑time AI applications fast and dependable, and unlocks new capabilities across enterprise and frontier AI use cases.
Why Join Cerebras
People who are serious about software make their own hardware. At Cerebras, we have built a breakthrough architecture that is unlocking new opportunities for the AI industry. With dozens of model releases and rapid growth, we’ve reached an inflection point in our business. Members of our team tell us there are five main reasons they joined Cerebras:
Build a breakthrough AI platform beyond the constraints of the GPU.
Publish and open source their cutting‑edge AI research.
Work on one of the fastest AI supercomputers in the world.
Enjoy job stability with startup vitality.
Our simple, non‑corporate work culture that respects individual beliefs.
Find out more about what it's like to work at Cerebras
here !
Cerebras Systems is committed to creating an equal and diverse environment and is proud to be an equal opportunity employer. We celebrate different backgrounds, perspectives, and skills. We believe inclusive teams build better products and companies. We try every day to build a work environment that empowers people to do their best work through continuous learning, growth and support of those around them.
This website or its third‑party tools process personal data. For more details, click here to review our CCPA disclosure notice.
#J-18808-Ljbffr
Vacancy posted 1 hour ago
Similar jobs that could be interesting for youBased on the Engineering Manager, Inference Router in Winnipeg, MB vacancy
  • Cerebras Systems is seeking an Engineering Manager, Inference Router to lead the team that directs where every inference request runs across Cerebras data centers and partners. You will own global routing, cache-aware scheduling, and autoscaling, partnering with ML and cloud... 
    Suggested

    Cerebras

    Winnipeg, MB
    1 hour ago
  • Elevate your career at Collabera as a Senior Software Engineer focused on network router infrastructure. Utilize your strong C programming skills in a collaborative environment. This position is full-time, where you will not only solve abstract problems but also mentor team... 
    Suggested
    Full time

    Collabera

    Winnipeg, MB
    1 hour ago
  • $165k - $330k per year

     ...Baseten Baseten powers mission-critical inference for the world's most dynamic AI...  ...Capital. Join us and help build the platform engineers turn to ship AI products.THE ROLE As...  ...development that also includes aspects of product management, technical customer success, and pre-... 
    Suggested
    Internship
    Flexible hours

    Urban Ridge Supplies

    Winnipeg, MB
    1 hour ago
  • Senior Software Engineer – Network Router Infrastructure Full-timeEstablished in 1991, Collabera has been a leader in IT staffing for over 22...  ...quality IT resources through Staff Augmentation, Global Talent Management, Value Added Services through CLASS (Competency Leveraged... 
    Suggested
    Permanent employment

    Collabera

    Winnipeg, MB
    1 hour ago
  • $165k - $330k per year

    About BasetenBaseten powers mission-critical inference for the world's most dynamic AI companies...  .... Join us and help build the platform engineers turn to ship AI products. THE ROLEAs...  ...the tools customers rely on every day to manage mission‑critical inference deployments.... 
    Suggested
    Flexible hours

    Baseten

    Winnipeg, MB
    1 hour ago
  •  ...to deliver industry-leading training and inference speeds; over 10 times faster than GPU-...  ...-fast decode on the Cerebras Wafer-Scale Engine. We are hiring a Software Engineer to...  ...upgrade, rollback, health-checking, capacity-management, and failure-recovery practices for the... 

    Cerebras

    Winnipeg, MB
    1 hour ago
  • $180k - $360k per year

     ...Baseten powers mission-critical inference for the world's most dynamic...  ...and help build the platform engineers turn to ship AI products....  ...Baseten multi-cloud capacity management (MCM) unifies deploymentsResponsibilities...  ...to LLM inference engines and frameworks such as vLLM,... 
    Flexible hours

    Baseten

    Winnipeg, MB
    1 hour ago
  • $100k - $145k per year

    Thomson Reuters is seeking a Senior Inference Engineer, AI. This person will collaborate with platform...  ..., OpenSearch vector search, API managed model routing). The successful candidate...  ...supportive workplace policies designed to help manage personal and professional... 
    Work at office
    Local area
    Flexible hours
    2 days per week
    3 days per week

    Thomson Reuters

    Winnipeg, MB
    1 hour ago
  • Elevate AI workloads with Baseten as an Inference Performance Engineer. Focus on optimizing cutting-edge inference techniques while working in a dynamic...  ...skills such as quantization, decoder specs, and KV-cache management. You’ll be directly impacting how our customers’ models... 

    Baseten

    Winnipeg, MB
    1 hour ago
  • $180k - $360k per year

     ...Baseten powers mission-critical inference for the world's most dynamic...  ...and help build the platform engineers turn to ship AI products....  ...the stack, from the inference engine and runtime through scheduling...  ...speculative decoding, and KV-cache management. You'll reason from first... 
    Flexible hours

    Baseten

    Winnipeg, MB
    1 hour ago
  •  ...Cohere is a team of researchers, engineers, designers, and more, who are...  ...ambiguous, customer-specific inference challenges—ranging from multi...  ...with modern inference engines and optimization stacks (e.g....  ...same-day production code while managing client-side deployment politics... 
    Full time
    Work at office
    Local area
    Remote work
    Home office

    FDE Times

    Winnipeg, MB
    1 hour ago
  •  ...architecture allows Cerebras to deliver industry-leading training and inference speeds; over 10 times faster than GPU-based hyperscale cloud...  ...high-speed inference. About the RoleWe're hiring a Software Engineer to help contribute to projects on our Inference Platform team.... 

    Cerebras

    Winnipeg, MB
    1 hour ago
  • Cohere is hiring a Backend Engineer to optimize model inference systems. You will build low-latency serving infrastructure, implement batching strategies, and develop the backend services that power Cohere's enterprise API.Requirements 5+ years of backend engineering experienceStrong... 
    Work at office

    Best AI Tools Wiki

    Winnipeg, MB
    1 hour ago
  • # Product Manager, Inference PlatformUnited States2 hours agoID 5413304Price on request## DetailsEmployment type: Full-timeRemote: YesCompany...  ...they drive for our customers. Cohere is a team of researchers, engineers, designers, and more, who are all passionate about their... 

    Bazeta

    Winnipeg, MB
    1 hour ago
  • Cohere is looking for a Backend Engineer to specialize in optimizing model inference systems. In this remote-first role, you will create low-latency serving infrastructure for advanced backend services. Candidates for this position must have over five years of backend engineering... 
    Remote work

    Best AI Tools Wiki

    Winnipeg, MB
    1 hour ago
  • Build AI inference systems with Python and PyTorch as a Backend Engineer, AI, powering production interactions through reliable, observable APIs.You will own the inference and orchestration layer between models and users. The work covers production backend systems, AI-powered... 

    Sarsfield Group

    Winnipeg, MB
    1 hour ago
  • Staff + Senior Software Engineer, InferenceAbout AnthropicAnthropic’s mission is to create...  ...AI systems. About the roleOur Inference team is responsible for building and maintaining...  ...routing, load balancing, and traffic management systems across thousands of accelerators... 
    Long term contract
    Work at office
    Worldwide
    Visa sponsorship
    Flexible hours

    Anthropic Limited

    Winnipeg, MB
    1 hour ago
  •  ...to deliver industry-leading training and inference speeds; over 10 times faster than GPU-...  ...About The RoleWe’re hiring a Senior Frontend Engineer to own and scale critical parts of the...  ...and enterprises use to run and manage inference workloads. This is a deeply... 
    Long term contract
    Shift work

    Cerebras

    Winnipeg, MB
    1 hour ago
  • Transform the AI experience at Cerebras Systems as a Software Engineer on our Inference Platform team. Shape next-gen architecture while deploying...  ...software across various domains • Define technical direction and manage platform roadmap • Architect high-availability systems with... 

    Cerebras

    Winnipeg, MB
    1 hour ago
  •  ...to deliver industry-leading training and inference speeds; over 10 times faster than GPU-...  ...-fast decode on the Cerebras Wafer-Scale Engine. We are hiring a Software Engineer to...  ..., error handling, retries, and lifecycle management. Improve serving performance. Optimize... 

    Cerebras

    Winnipeg, MB
    1 hour ago
  • Build machine learning inference APIs with Python and distributed systems expertise at Cerebras Systems, connecting GPU prefill with Cerebras decode.As a Senior Software Engineer, you will develop production software across inference APIs, model integration, and request-routing... 

    FDE Times

    Winnipeg, MB
    1 hour ago
  • Senior AI Engineer, AI Inference Systems Join to apply for the Senior AI Engineer, AI Inference Systems role at Brix .Location : Palo...  ...EquityResponsibilities Oversee and optimize the LLM inference engine to ensure performance, scalability, and cost-efficiency.... 

    Brix

    Winnipeg, MB
    1 hour ago
  • Shape AI inference software at Cerebras Systems with Python, model serving, and production API experience across accelerator backends.This Senior Software Engineer role focuses on making disaggregated inference accessible, reliable, and consistent. You will work across model... 

    Cerebras

    Winnipeg, MB
    1 hour ago
  •  ...We’re hiring a Staff Backend Engineer to build and operate the infrastructure...  ..., and scalability of inference systems. Join us and help influence...  ...Prometheus and Grafana Manage and optimize cloud...  ...] is the world’s leading game engine, powering play for more than 3... 
    Work at office
    Worldwide

    Unity Technologies

    Winnipeg, MB
    1 hour ago
  • A leading technology company in Markham, Canada, seeks a Principal Software Development Engineer focused on generative AI training and inference. You will collaborate with a talented team to innovate efficient techniques for model scaling. Required qualifications include a relevant... 

    Advanced Micro Devices, Inc.

    Winnipeg, MB
    1 hour ago
  •  ...a highly experienced Back-End Software Engineer who wants to work on core systems that shape...  ...-shelf solutionsModel Vault is Cohere’s inference platform for deploying and operating...  ...authentication and authorization, deployment management, billing, metering, and customer... 
    Remote work

    Cohere

    Winnipeg, MB
    1 hour ago
  •  ...looking for a Senior Software Development Engineer to own the end-to-end model execution...  ...infrastructure at scale and high-performance inference serving. The PersonThis role demands...  ...AMD GPUs: batching strategies, KV-cache management, speculative decoding, and continuous... 
    Internship
    Manual labor

    Advanced Micro Devices, Inc.

    Winnipeg, MB
    1 hour ago
  • Cohere is seeking an experienced engineer to join the Model Efficiency team, focusing on building reliable ML systems and optimizing LLM inference. You will improve the inference stack, reduce latency, and boost throughput across diverse workloads. The role offers remote-friendly... 
    Remote work

    Cohere

    Winnipeg, MB
    1 hour ago
  • Enhance AI capabilities as a Senior Inference Engineer with Thomson Reuters, where hybrid work enables flexibility. Collaborate with cross-functional teams to deploy and scale cutting-edge AI workloads. In this influential role, you will be embedded within Platform Engineering... 

    Thomson Reuters

    Winnipeg, MB
    1 hour ago
  •  ...Cerebras Systems builds the world's largest AI chip and is hiring a Software Engineer to build and evolve the ML API layer for a disaggregated inference system. You will work across inference APIs, model integration, request-routing services, and Cerebras runtimes to deliver... 

    Cerebras

    Winnipeg, MB
    1 hour ago

Do you want to receive more vacancies?

Subscribe and receive similar vacancies to Engineering Manager, Inference Router. Be the first to apply!