Sign up to access all features of our service.
  • Job search
  • Favorites
  • Create a CV
    New
  • Salaries
  • Subscriptions

Staff Software Engineer, Inference Infrastructure

Cohere

Who are we?Our mission is to scale intelligence to serve humanity. We’re training and deploying frontier models for developers and enterprises who are building AI systems to power magical experiences like content generation, semantic search, RAG, and agents. We believe that our work is instrumental to the widespread adoption of AI. We obsess over what we build. Each one of us is responsible for contributing to increasing the capabilities of our models and the value they drive for our customers. We like to work hard and move fast to do what’s best for our customers. Cohere is a team of researchers, engineers, designers, and more, who are passionate about their craft. Each person is one of the best in the world at what they do. We believe that a diverse range of perspectives is a requirement for building great products. Join us on our mission and shape the future!
Why this role?Are you energized by building high-performance, scalable and reliable machine learning systems? Do you want to help define and build the next generation of AI platforms powering advanced NLP applications? We are looking for Members of Technical Staff to join the Model Serving team at Cohere. The team is responsible for developing, deploying, and operating the AI platform delivering Cohere's large language models through easy to use API endpoints. In this role, you will work closely with many teams to deploy optimized NLP models to production in low latency, high throughput, and high availability environments. You will also get the opportunity to interface with customers and create customized deployments to meet their specific needs. You may be a good fit if you have:
5+ years of engineering experience running production infrastructure at a large scale
Experience designing large, highly available distributed systems with Kubernetes, and GPU workloads on those clusters
Experience with Kubernetes dev and production coding and support
Experience with GCP, Azure, AWS, OCI, multi-cloud on-prem / hybrid serving
Experience in designing, deploying, supporting, and troubleshooting in complex Linux-based computing environments
Experience in compute/storage/network resource and cost management
Excellent collaboration and troubleshooting skills to build mission-critical systems, and ensure smooth operations and efficient teamwork
The grit and adaptability to solve complex technical challenges that evolve day to day
Familiarity with computational characteristics of accelerators (GPUs, TPUs, and/or custom accelerators), especially how they influence latency and throughput of inference.
Strong understanding or working experience with distributed systems.
Experience in Golang, C++ or other languages designed for high-performance scalable servers).
We value and celebrate diversity and strive to create an inclusive work environment for all. We welcome applicants from all backgrounds and are committed to providing equal opportunities. Should you require any accommodations during the recruitment process, please submit an Accommodations Request Form, and we will work together to meet your needs.
Full-Time Employees at Cohere enjoy these Perks:
An open and inclusive culture and work environment
Work closely with a team on the cutting edge of AI research
Weekly lunch stipend, in-office lunches & snacks
Full health and dental benefits, including a separate budget to take care of your mental health
100% Parental Leave top-up for up to 6 months
Personal enrichment benefits towards arts and culture, fitness and well-being, quality time, and workspace improvement
Remote-flexible, offices in Toronto, New York, San Francisco, London and Paris, as well as a co-working stipend
6 weeks of vacation (30 working days!)
#J-18808-Ljbffr
Vacancy posted 45 minutes ago
Similar jobs that could be interesting for youBased on the Staff Software Engineer, Inference Infrastructure in Winnipeg, MB vacancy
  • Staff + Senior Software Engineer, InferenceAbout AnthropicAnthropic’s mission is to create reliable, interpretable, and steerable AI systems. We want...  ...together to build beneficial AI systems. About the roleOur Inference team is responsible for building and maintaining the... 
    Suggested
    Long term contract
    Work at office
    Worldwide
    Visa sponsorship
    Flexible hours

    Anthropic Limited

    Winnipeg, MB
    35 minutes ago
  •  ...deliver industry-leading training and inference speeds; over 10 times faster...  ...decode on the Cerebras Wafer-Scale Engine. We are hiring a Software Engineer to productionize and optimize...  ...stack, and rack-scale AMD GPU infrastructure, to make this new serving path reliable... 
    Suggested

    Cerebras

    Winnipeg, MB
    46 minutes ago
  •  ...deliver industry-leading training and inference speeds; over 10 times faster...  ...decode on the Cerebras Wafer-Scale Engine. We are hiring a Software Engineer to build and evolve the ML...  ...enablement, compiler, runtime, cloud infrastructure, product, customer-facing teams,... 
    Suggested

    Cerebras

    Winnipeg, MB
    45 minutes ago
  •  ...to deliver industry-leading training and inference speeds; over 10 times faster than GPU...  .... About the RoleWe're hiring a Staff Engineer to help lead, drive, and contribute to projects...  ...develop, test, and maintain production software, with responsibilities spanning testing,... 
    Suggested

    Cerebras

    Winnipeg, MB
    35 minutes ago
  •  ...Cerebras Systems builds the world's largest AI chip and is hiring a Software Engineer to build and evolve the ML API layer for a disaggregated inference system. You will work across inference APIs, model integration, request-routing services, and Cerebras runtimes to deliver... 
    Suggested

    Cerebras

    Winnipeg, MB
    39 minutes ago
  •  ...is for a highly experienced Back-End Software Engineer who wants to work on core systems that...  ...shelf solutionsModel Vault is Cohere’s inference platform for deploying and operating models...  ...a customer and the underlying AI infrastructure, including identity, deployments, metering... 
    Remote work

    Cohere

    Winnipeg, MB
    44 minutes ago
  • $176.5k per year

     ...About The Team And Role Core Infrastructure builds and operates the foundations...  ...content that powers our products and our inference‑driven features. The core of the...  ...will join an existing team of Senior, Staff, and Principal engineers building the Content Library and... 
    Long term contract
    For contractors
    Local area
    Home office
    Flexible hours

    Scribd, Inc.

    Winnipeg, MB
    44 minutes ago
  •  ...industry-leading training and inference speeds; over 10 times faster...  ...monitor, and optimize cloud infrastructure supporting CI workloads,...  ...increase developer velocity and engineering productivity. Identify...  ...professional experience in software engineering, infrastructure... 
    Long term contract

    Cerebras

    Winnipeg, MB
    46 minutes ago
  •  ...customers include some of the world’s largest airlines and ground handling providers. For more information, visit As a Staff Software Engineer, Infrastructure (Cloud), you will own the data infrastructure and cloud systems that power AeroVect's autonomous driving platform.... 
    Long term contract

    AeroVect

    Winnipeg, MB
    40 minutes ago
  •  ...customers. Cohere is a team of researchers, engineers, designers, and more, who are all...  ...building the next generation of agentic AI infrastructure at Cohere. This team sits at the intersection...  ...in emerging ML infrastructure, edge inference, or browser‑native models Open‑source... 
    Full time
    Manual labor
    Work at office
    Local area
    Remote work
    Home office

    FDE Times

    Winnipeg, MB
    37 minutes ago
  • About the Role Software engineering is undergoing its biggest transformation in decades.AI agents...  ...the next.We’re looking for a Senior Staff Engineer to help define what that...  ...pipelines to agent sandboxes, testing infrastructure, repository architecture, and AI-native... 
    Long term contract
    Internship
    Work at office
    Local area
    3 days per week

    Rippling

    Winnipeg, MB
    46 minutes ago
  • $165k - $330k per year

     ...BasetenBaseten powers mission-critical inference for the world's most dynamic AI companies...  ...By uniting applied AI research, flexible infrastructure, and seamless developer tooling, we...  ...Capital. Join us and help build the platform engineers turn to ship AI products. THE ROLEAs... 
    Flexible hours

    Baseten

    Winnipeg, MB
    36 minutes ago
  • $180k - $360k per year

     ...Baseten powers mission-critical inference for the world's most dynamic...  ...AI research, flexible infrastructure, and seamless developer tooling...  ...and help build the platform engineers turn to ship AI products....  ...enhancing the performance of software systems, particularly in the... 
    Flexible hours

    Baseten

    Winnipeg, MB
    35 minutes ago
  • $180k - $360k per year

     ...Baseten powers mission-critical inference for the world's most dynamic...  ...AI research, flexible infrastructure, and seamless developer tooling...  ...and help build the platform engineers turn to ship AI products.About...  ...contributions to LLM inference engines and frameworks such as vLLM,... 
    Flexible hours

    Baseten

    Winnipeg, MB
    41 minutes ago
  • $181k - $241k per year

    Staff Software Engineer, Mobile (App Infrastructure)Remote Canada Affirm is reinventing credit to make it more honest and friendly, giving consumers the flexibility to buy now and pay later without any hidden fees or compounding interest. App Infrastructure is the team... 
    Work at office
    Remote work
    Flexible hours

    Affirm, Inc.

    Winnipeg, MB
    35 minutes ago
  • Transform the AI experience at Cerebras Systems as a Software Engineer on our Inference Platform team. Shape next-gen architecture while deploying...  ...distributed systems • Experience with Kubernetes and cloud infrastructure • Strong knowledge of backend languages like Go or C++ •... 

    Cerebras

    Winnipeg, MB
    37 minutes ago
  •  ..., shaping architecture, and mentoring engineers to do their best work and advance their...  ...into how we build, test, and operate infrastructure software. This is not a 'stay in your swim lane...  ...GPU infrastructure for training and inference. Here's what your benefits package... 
    Local area
    Worldwide
    Flexible hours

    DataRobot

    Winnipeg, MB
    42 minutes ago
  •  ...industry-leading training and inference speeds; over 10 times faster...  ...of secure, scalable cloud infrastructure and identity platforms in AWS...  .... Partner across Security, Engineering, and Infrastructure teams to...  ...People who are serious about software make their own hardware. At... 

    Cerebras

    Winnipeg, MB
    46 minutes ago
  •  ...world impact converge at scale. We’re hiring a Staff Backend Engineer to build and operate the infrastructure those models depend on. You’ll design and operate...  ...the performance, reliability, and scalability of inference systems. Join us and help influence how billions... 
    Work at office
    Worldwide

    Unity Technologies

    Winnipeg, MB
    34 minutes ago
  • $147k - $196.6k per year

     ...We are seeking a highly skilled and motivated Staff Software Developer to join our Diagnostics Infrastructure Software team in the Autonomous Vehicle Integrity...  .... Your work will be conducted in a collaborative engineering environment, where you will exercise the highest... 
    Long term contract
    Work at office

    General Motors Company

    Winnipeg, MB
    39 minutes ago
  • As the Senior Staff Software Developer, Developer Infrastructure, you are the technical driver of this entire org — the Uber Tech Lead unifying DevX, TestX,...  ...pipelines to agentic coding systems that multiply what each engineer can ship This is a rare opportunity to shape not... 
    Long term contract
    Remote work

    Robinhood

    Winnipeg, MB
    46 minutes ago
  • $180k - $210k per year

     ...applications and next steps. Our partner is looking for a Staff Network Engineer (AI Infrastructure) based in Canada. The Staff Network Engineer will...  ...fabrics for large-scale AI workloads. You will develop software-driven solutions that configure network switches, maintain... 

    Jobgether

    Winnipeg, MB
    45 minutes ago
  • $165k - $330k per year

     ...Baseten Baseten powers mission-critical inference for the world's most dynamic AI...  ...By uniting applied AI research, flexible infrastructure, and seamless developer tooling, we enable...  ...Capital. Join us and help build the platform engineers turn to ship AI products. Baseten... 
    Long term contract
    Flexible hours

    Baseten

    Winnipeg, MB
    46 minutes ago
  • $165k - $330k per year

     ...BasetenBaseten powers mission-critical inference for the world's most dynamic...  ...AI research, flexible infrastructure, and seamless developer...  ...and help build the platform engineers turn to ship AI products....  ...THE ROLEAs an Infrastructure Software Engineer at Baseten, you'll... 
    Flexible hours

    Baseten

    Winnipeg, MB
    36 minutes ago
  • Cerebras Systems is hiring a Software Engineer to productionize and optimize our GPU serving stack. You will work across custom inference APIs, vLLM, PyTorch, ROCm, and rack-scale AMD GPU infrastructure to ensure reliability, numerical correctness, observability, and high performance... 

    FDE Times

    Winnipeg, MB
    37 minutes ago
  • $200k - $250k per year

     ..., automating everything from engineering workflows to regulatory filings...  ...at the heart of how critical infrastructure is planned, monitored, and...  ...The RoleWe're looking for a staff engineer to define and drive...  ...building and shipping production software, including experience... 
    Work at office

    Senpilot

    Winnipeg, MB
    37 minutes ago
  • $167.55k - $209.44k per year

     ...canonical and ready to query, with no engineering or maintenance required. We’re proud that...  ....Together, we’re delivering the data infrastructure layer that helps organizations move,...  ...companies. We’re seeking an enthusiastic Staff Software Engineer to join our fast-growing data... 
    Full time
    Internship
    Work at office
    Remote work
    Flexible hours

    Fivetran, Inc.

    Winnipeg, MB
    41 minutes ago
  •  ...decisions get made, and how AI-augmented engineering works in a regulated environment....  ...Significant experiencebuilding consumer-facing software at scale, with architectural judgment...  ...move between frontend, backend, and infrastructure when the problem demands it.Experience... 
    Internship
    Work at office
    Work from home
    Flexible hours

    Purpose Unlimited

    Winnipeg, MB
    44 minutes ago
  • $140k - $190k per year

     ...development of digital investigative software that acquires, analyzes, and shares evidence...  ...We’re looking for a seasoned Staff Software Engineer who thrives on technical depth and...  ....Nice To Have Experience with Infrastructure as Code (Terraform, Helm, Argo).Experience... 
    Work at office
    Local area
    Flexible hours

    Magnet Forensics

    Winnipeg, MB
    45 minutes ago
  •  ...starts with a query. Our in-house OLAP engine, Nova , processes trillions of events...  .... That makes Nova the critical infrastructure in the loop, and as non-stop agents become...  ...grows dramatically. We’re looking for a Staff Software Engineer who wants to go deep on both... 
    Immediate start
    Worldwide

    Amplitude

    Winnipeg, MB
    35 minutes ago

Do you want to receive more vacancies?

Subscribe and receive similar vacancies to Staff Software Engineer, Inference Infrastructure. Be the first to apply!