Sign up to access all features of our service.
  • Job search
  • Favorites
  • Create a CV
    New
  • Salaries
  • Subscriptions

Staff Software Engineer, Inference Infrastructure

Cohere

Who are we?Our mission is to scale intelligence to serve humanity. We’re training and deploying frontier models for developers and enterprises who are building AI systems to power magical experiences like content generation, semantic search, RAG, and agents. We believe that our work is instrumental to the widespread adoption of AI. We obsess over what we build. Each one of us is responsible for contributing to increasing the capabilities of our models and the value they drive for our customers. We like to work hard and move fast to do what’s best for our customers. Cohere is a team of researchers, engineers, designers, and more, who are passionate about their craft. Each person is one of the best in the world at what they do. We believe that a diverse range of perspectives is a requirement for building great products. Join us on our mission and shape the future!
Why this role?Are you energized by building high-performance, scalable and reliable machine learning systems? Do you want to help define and build the next generation of AI platforms powering advanced NLP applications? We are looking for Members of Technical Staff to join the Model Serving team at Cohere. The team is responsible for developing, deploying, and operating the AI platform delivering Cohere's large language models through easy to use API endpoints. In this role, you will work closely with many teams to deploy optimized NLP models to production in low latency, high throughput, and high availability environments. You will also get the opportunity to interface with customers and create customized deployments to meet their specific needs. You may be a good fit if you have:
5+ years of engineering experience running production infrastructure at a large scale
Experience designing large, highly available distributed systems with Kubernetes, and GPU workloads on those clusters
Experience with Kubernetes dev and production coding and support
Experience with GCP, Azure, AWS, OCI, multi-cloud on-prem / hybrid serving
Experience in designing, deploying, supporting, and troubleshooting in complex Linux-based computing environments
Experience in compute/storage/network resource and cost management
Excellent collaboration and troubleshooting skills to build mission-critical systems, and ensure smooth operations and efficient teamwork
The grit and adaptability to solve complex technical challenges that evolve day to day
Familiarity with computational characteristics of accelerators (GPUs, TPUs, and/or custom accelerators), especially how they influence latency and throughput of inference.
Strong understanding or working experience with distributed systems.
Experience in Golang, C++ or other languages designed for high-performance scalable servers).
We value and celebrate diversity and strive to create an inclusive work environment for all. We welcome applicants from all backgrounds and are committed to providing equal opportunities. Should you require any accommodations during the recruitment process, please submit an Accommodations Request Form, and we will work together to meet your needs.
Full-Time Employees at Cohere enjoy these Perks:
An open and inclusive culture and work environment
Work closely with a team on the cutting edge of AI research
Weekly lunch stipend, in-office lunches & snacks
Full health and dental benefits, including a separate budget to take care of your mental health
100% Parental Leave top-up for up to 6 months
Personal enrichment benefits towards arts and culture, fitness and well-being, quality time, and workspace improvement
Remote-flexible, offices in Toronto, New York, San Francisco, London and Paris, as well as a co-working stipend
6 weeks of vacation (30 working days!)
#J-18808-Ljbffr
Vacancy posted 4 hours ago
Similar jobs that could be interesting for youBased on the Staff Software Engineer, Inference Infrastructure in Winnipeg, MB vacancy
  •  ...to deliver industry-leading training and inference speeds; over 10 times faster than GPU...  .... About the RoleWe're hiring a Staff Engineer to help lead, drive, and contribute to projects...  ...develop, test, and maintain production software, with responsibilities spanning testing,... 
    Suggested

    Cerebras

    Winnipeg, MB
    4 hours ago
  •  ...deliver industry-leading training and inference speeds; over 10 times faster...  ...decode on the Cerebras Wafer-Scale Engine. We are hiring a Software Engineer to productionize and optimize...  ...stack, and rack-scale AMD GPU infrastructure, to make this new serving path reliable... 
    Suggested

    Cerebras

    Winnipeg, MB
    4 hours ago
  • Staff + Senior Software Engineer, InferenceAbout AnthropicAnthropic’s mission is to create reliable, interpretable, and steerable AI systems. We want...  ...together to build beneficial AI systems. About the roleOur Inference team is responsible for building and maintaining the... 
    Suggested
    Long term contract
    Work at office
    Worldwide
    Visa sponsorship
    Flexible hours

    Anthropic Limited

    Winnipeg, MB
    4 hours ago
  •  ...deliver industry-leading training and inference speeds; over 10 times faster...  ...decode on the Cerebras Wafer-Scale Engine. We are hiring a Software Engineer to build and evolve the ML...  ...enablement, compiler, runtime, cloud infrastructure, product, customer-facing teams,... 
    Suggested

    Cerebras

    Winnipeg, MB
    4 hours ago
  •  ...is for a highly experienced Back-End Software Engineer who wants to work on core systems that...  ...shelf solutionsModel Vault is Cohere’s inference platform for deploying and operating models...  ...a customer and the underlying AI infrastructure, including identity, deployments, metering... 
    Suggested
    Remote work

    Cohere

    Winnipeg, MB
    4 hours ago
  •  ...Cerebras Systems builds the world's largest AI chip and is hiring a Software Engineer to build and evolve the ML API layer for a disaggregated inference system. You will work across inference APIs, model integration, request-routing services, and Cerebras runtimes to deliver... 

    Cerebras

    Winnipeg, MB
    4 hours ago
  • $176.5k per year

     ...About The Team And Role Core Infrastructure builds and operates the foundations...  ...content that powers our products and our inference‑driven features. The core of the...  ...will join an existing team of Senior, Staff, and Principal engineers building the Content Library and... 
    Long term contract
    For contractors
    Local area
    Home office
    Flexible hours

    Scribd, Inc.

    Winnipeg, MB
    4 hours ago
  •  ...customers include some of the world’s largest airlines and ground handling providers. For more information, visit As a Staff Software Engineer, Infrastructure (Cloud), you will own the data infrastructure and cloud systems that power AeroVect's autonomous driving platform.... 
    Long term contract

    AeroVect

    Winnipeg, MB
    4 hours ago
  •  ...industry-leading training and inference speeds; over 10 times faster...  ...monitor, and optimize cloud infrastructure supporting CI workloads,...  ...increase developer velocity and engineering productivity. Identify...  ...professional experience in software engineering, infrastructure... 
    Long term contract

    Cerebras

    Winnipeg, MB
    4 hours ago
  •  ...customers. Cohere is a team of researchers, engineers, designers, and more, who are all...  ...building the next generation of agentic AI infrastructure at Cohere. This team sits at the intersection...  ...in emerging ML infrastructure, edge inference, or browser‑native modelsOpen‑source... 
    Work at office
    Local area
    Remote work
    Home office

    Cohere

    Winnipeg, MB
    4 hours ago
  • About the Role Software engineering is undergoing its biggest transformation in decades.AI agents...  ...the next.We’re looking for a Senior Staff Engineer to help define what that...  ...pipelines to agent sandboxes, testing infrastructure, repository architecture, and AI-native... 
    Long term contract
    Internship
    Work at office
    Local area
    3 days per week

    Rippling

    Winnipeg, MB
    4 hours ago
  • $180k - $360k per year

     ...Baseten powers mission-critical inference for the world's most dynamic...  ...AI research, flexible infrastructure, and seamless developer tooling...  ...and help build the platform engineers turn to ship AI products.About...  ...contributions to LLM inference engines and frameworks such as vLLM,... 
    Flexible hours

    Baseten

    Winnipeg, MB
    4 hours ago
  • $180k - $360k per year

     ...Baseten powers mission-critical inference for the world's most dynamic...  ...AI research, flexible infrastructure, and seamless developer tooling...  ...and help build the platform engineers turn to ship AI products....  ...enhancing the performance of software systems, particularly in the... 
    Flexible hours

    Baseten

    Winnipeg, MB
    4 hours ago
  • $165k - $330k per year

     ...BasetenBaseten powers mission-critical inference for the world's most dynamic AI companies...  ...By uniting applied AI research, flexible infrastructure, and seamless developer tooling, we...  ...Capital. Join us and help build the platform engineers turn to ship AI products. THE ROLEAs... 
    Flexible hours

    Baseten

    Winnipeg, MB
    4 hours ago
  • $181k - $241k per year

    Staff Software Engineer, Mobile (App Infrastructure)Remote Canada Affirm is reinventing credit to make it more honest and friendly, giving consumers the flexibility to buy now and pay later without any hidden fees or compounding interest. App Infrastructure is the team... 
    Work at office
    Remote work
    Flexible hours

    Affirm, Inc.

    Winnipeg, MB
    4 hours ago
  • Shape AI inference software at Cerebras Systems with Python, model serving, and production API...  ...accelerator backends.This Senior Software Engineer role focuses on making disaggregated...  ...service, framework, runtime, or infrastructure boundariesFamiliarity with REST, gRPC,... 

    Cerebras

    Winnipeg, MB
    4 hours ago
  •  ..., shaping architecture, and mentoring engineers to do their best work and advance their...  ...into how we build, test, and operate infrastructure software. This is not a 'stay in your swim lane...  ...GPU infrastructure for training and inference. Here's what your benefits package... 
    Local area
    Worldwide
    Flexible hours

    DataRobot

    Winnipeg, MB
    4 hours ago
  •  ...industry-leading training and inference speeds; over 10 times faster...  ...of secure, scalable cloud infrastructure and identity platforms in AWS...  .... Partner across Security, Engineering, and Infrastructure teams to...  ...People who are serious about software make their own hardware. At... 

    Cerebras

    Winnipeg, MB
    4 hours ago
  •  ...world impact converge at scale. We’re hiring a Staff Backend Engineer to build and operate the infrastructure those models depend on. You’ll design and operate...  ...the performance, reliability, and scalability of inference systems. Join us and help influence how billions... 
    Work at office
    Worldwide

    Unity Technologies

    Winnipeg, MB
    4 hours ago
  • Transform the AI experience at Cerebras Systems as a Software Engineer on our Inference Platform team. Shape next-gen architecture while deploying...  ...distributed systems • Experience with Kubernetes and cloud infrastructure • Strong knowledge of backend languages like Go or C++ •... 

    Cerebras

    Winnipeg, MB
    4 hours ago
  • $165k - $330k per year

     ...BasetenBaseten powers mission-critical inference for the world's most dynamic...  ...AI research, flexible infrastructure, and seamless developer...  ...and help build the platform engineers turn to ship AI products....  ...THE ROLEAs an Infrastructure Software Engineer at Baseten, you'll... 
    Flexible hours

    Baseten

    Winnipeg, MB
    4 hours ago
  • $165k - $330k per year

     ...Baseten Baseten powers mission-critical inference for the world's most dynamic AI...  ...By uniting applied AI research, flexible infrastructure, and seamless developer tooling, we enable...  ...Capital. Join us and help build the platform engineers turn to ship AI products. Baseten... 
    Long term contract
    Flexible hours

    Baseten

    Winnipeg, MB
    4 hours ago
  • As the Senior Staff Software Developer, Developer Infrastructure, you are the technical driver of this entire org — the Uber Tech Lead unifying DevX, TestX,...  ...pipelines to agentic coding systems that multiply what each engineer can ship This is a rare opportunity to shape not... 
    Long term contract
    Remote work

    Robinhood

    Winnipeg, MB
    4 hours ago
  • $147k - $196.6k per year

     ...We are seeking a highly skilled and motivated Staff Software Developer to join our Diagnostics Infrastructure Software team in the Autonomous Vehicle Integrity...  .... Your work will be conducted in a collaborative engineering environment, where you will exercise the highest... 
    Long term contract
    Work at office

    General Motors Company

    Winnipeg, MB
    4 hours ago
  • Cerebras Systems is hiring a Software Engineer to productionize and optimize our GPU serving stack. You will work across custom inference APIs, vLLM, PyTorch, ROCm, and rack-scale AMD GPU infrastructure to ensure reliability, numerical correctness, observability, and high performance... 

    FDE Times

    Winnipeg, MB
    4 hours ago
  • $145k - $260k per year

    About VGSVGS is building the trust infrastructure for a new era of commerce. As commerce becomes...  ...takes exceptional people. Trusted Engineering That Matters Engineering at VGS VGS...  ...reliability practices. About the RoleAs a Staff Infrastructure Engineer, you will serve... 
    Work at office
    Local area
    Immediate start
    Remote work
    Flexible hours
    Shift work

    VGS

    Winnipeg, MB
    4 hours ago
  • $180k - $210k per year

     ...applications and next steps. Our partner is looking for a Staff Network Engineer (AI Infrastructure) based in Canada. The Staff Network Engineer will...  ...fabrics for large-scale AI workloads. You will develop software-driven solutions that configure network switches, maintain... 
    Local area
    Remote work
    Flexible hours

    Jobgether

    Winnipeg, MB
    4 hours ago
  • $200k - $250k per year

     ..., automating everything from engineering workflows to regulatory filings...  ...at the heart of how critical infrastructure is planned, monitored, and...  ...The RoleWe're looking for a staff engineer to define and drive...  ...building and shipping production software, including experience... 
    Work at office

    Senpilot

    Winnipeg, MB
    4 hours ago
  • $140k - $190k per year

     ...development of digital investigative software that acquires, analyzes, and shares evidence...  ...We’re looking for a seasoned Staff Software Engineer who thrives on technical depth and...  ....Nice To Have Experience with Infrastructure as Code (Terraform, Helm, Argo).Experience... 
    Work at office
    Local area
    Flexible hours

    Magnet Forensics

    Winnipeg, MB
    4 hours ago
  • # Staff Software EngineerJuniper Square190k–235k USDFull-timeremote· leadCanada, USApythonnodejavareactawsdockerkubernetesListed...  ....About your roleAs a Staff Software Engineer at Juniper Square, you are a technical...  ...equivalent), Docker, Kubernetes, and infrastructure as code-Strong understanding of CI/CD,... 
    Full time
    Temporary work
    Work at office
    Remote work

    Devconnectplatform

    Winnipeg, MB
    4 hours ago

Do you want to receive more vacancies?

Subscribe and receive similar vacancies to Staff Software Engineer, Inference Infrastructure. Be the first to apply!