Staff Software Engineer, Inference Infrastructure
Cohere
Who are we?Our mission is to scale intelligence to serve humanity. We’re training and deploying frontier models for developers and enterprises who are building AI systems to power magical experiences like content generation, semantic search, RAG, and agents. We believe that our work is instrumental to the widespread adoption of AI. We obsess over what we build. Each one of us is responsible for contributing to increasing the capabilities of our models and the value they drive for our customers. We like to work hard and move fast to do what’s best for our customers. Cohere is a team of researchers, engineers, designers, and more, who are passionate about their craft. Each person is one of the best in the world at what they do. We believe that a diverse range of perspectives is a requirement for building great products. Join us on our mission and shape the future!
Why this role?Are you energized by building high-performance, scalable and reliable machine learning systems? Do you want to help define and build the next generation of AI platforms powering advanced NLP applications? We are looking for Members of Technical Staff to join the Model Serving team at Cohere. The team is responsible for developing, deploying, and operating the AI platform delivering Cohere's large language models through easy to use API endpoints. In this role, you will work closely with many teams to deploy optimized NLP models to production in low latency, high throughput, and high availability environments. You will also get the opportunity to interface with customers and create customized deployments to meet their specific needs. You may be a good fit if you have:
5+ years of engineering experience running production infrastructure at a large scale
Experience designing large, highly available distributed systems with Kubernetes, and GPU workloads on those clusters
Experience with Kubernetes dev and production coding and support
Experience with GCP, Azure, AWS, OCI, multi-cloud on-prem / hybrid serving
Experience in designing, deploying, supporting, and troubleshooting in complex Linux-based computing environments
Experience in compute/storage/network resource and cost management
Excellent collaboration and troubleshooting skills to build mission-critical systems, and ensure smooth operations and efficient teamwork
The grit and adaptability to solve complex technical challenges that evolve day to day
Familiarity with computational characteristics of accelerators (GPUs, TPUs, and/or custom accelerators), especially how they influence latency and throughput of inference.
Strong understanding or working experience with distributed systems.
Experience in Golang, C++ or other languages designed for high-performance scalable servers).
We value and celebrate diversity and strive to create an inclusive work environment for all. We welcome applicants from all backgrounds and are committed to providing equal opportunities. Should you require any accommodations during the recruitment process, please submit an Accommodations Request Form, and we will work together to meet your needs.
Full-Time Employees at Cohere enjoy these Perks:
An open and inclusive culture and work environment
Work closely with a team on the cutting edge of AI research
Weekly lunch stipend, in-office lunches & snacks
Full health and dental benefits, including a separate budget to take care of your mental health
100% Parental Leave top-up for up to 6 months
Personal enrichment benefits towards arts and culture, fitness and well-being, quality time, and workspace improvement
Remote-flexible, offices in Toronto, New York, San Francisco, London and Paris, as well as a co-working stipend
6 weeks of vacation (30 working days!)
#J-18808-Ljbffr
Why this role?Are you energized by building high-performance, scalable and reliable machine learning systems? Do you want to help define and build the next generation of AI platforms powering advanced NLP applications? We are looking for Members of Technical Staff to join the Model Serving team at Cohere. The team is responsible for developing, deploying, and operating the AI platform delivering Cohere's large language models through easy to use API endpoints. In this role, you will work closely with many teams to deploy optimized NLP models to production in low latency, high throughput, and high availability environments. You will also get the opportunity to interface with customers and create customized deployments to meet their specific needs. You may be a good fit if you have:
5+ years of engineering experience running production infrastructure at a large scale
Experience designing large, highly available distributed systems with Kubernetes, and GPU workloads on those clusters
Experience with Kubernetes dev and production coding and support
Experience with GCP, Azure, AWS, OCI, multi-cloud on-prem / hybrid serving
Experience in designing, deploying, supporting, and troubleshooting in complex Linux-based computing environments
Experience in compute/storage/network resource and cost management
Excellent collaboration and troubleshooting skills to build mission-critical systems, and ensure smooth operations and efficient teamwork
The grit and adaptability to solve complex technical challenges that evolve day to day
Familiarity with computational characteristics of accelerators (GPUs, TPUs, and/or custom accelerators), especially how they influence latency and throughput of inference.
Strong understanding or working experience with distributed systems.
Experience in Golang, C++ or other languages designed for high-performance scalable servers).
We value and celebrate diversity and strive to create an inclusive work environment for all. We welcome applicants from all backgrounds and are committed to providing equal opportunities. Should you require any accommodations during the recruitment process, please submit an Accommodations Request Form, and we will work together to meet your needs.
Full-Time Employees at Cohere enjoy these Perks:
An open and inclusive culture and work environment
Work closely with a team on the cutting edge of AI research
Weekly lunch stipend, in-office lunches & snacks
Full health and dental benefits, including a separate budget to take care of your mental health
100% Parental Leave top-up for up to 6 months
Personal enrichment benefits towards arts and culture, fitness and well-being, quality time, and workspace improvement
Remote-flexible, offices in Toronto, New York, San Francisco, London and Paris, as well as a co-working stipend
6 weeks of vacation (30 working days!)
#J-18808-Ljbffr
Vacancy posted 4 hours ago
Similar jobs that could be interesting for youBased on the Staff Software Engineer, Inference Infrastructure in Winnipeg, MB vacancy
- ...to deliver industry-leading training and inference speeds; over 10 times faster than GPU... .... About the RoleWe're hiring a Staff Engineer to help lead, drive, and contribute to projects... ...develop, test, and maintain production software, with responsibilities spanning testing,...Suggested
- ...deliver industry-leading training and inference speeds; over 10 times faster... ...decode on the Cerebras Wafer-Scale Engine. We are hiring a Software Engineer to productionize and optimize... ...stack, and rack-scale AMD GPU infrastructure, to make this new serving path reliable...Suggested
- Staff + Senior Software Engineer, InferenceAbout AnthropicAnthropic’s mission is to create reliable, interpretable, and steerable AI systems. We want... ...together to build beneficial AI systems. About the roleOur Inference team is responsible for building and maintaining the...SuggestedLong term contractWork at officeWorldwideVisa sponsorshipFlexible hours
- ...deliver industry-leading training and inference speeds; over 10 times faster... ...decode on the Cerebras Wafer-Scale Engine. We are hiring a Software Engineer to build and evolve the ML... ...enablement, compiler, runtime, cloud infrastructure, product, customer-facing teams,...Suggested
- ...is for a highly experienced Back-End Software Engineer who wants to work on core systems that... ...shelf solutionsModel Vault is Cohere’s inference platform for deploying and operating models... ...a customer and the underlying AI infrastructure, including identity, deployments, metering...SuggestedRemote work
- ...Cerebras Systems builds the world's largest AI chip and is hiring a Software Engineer to build and evolve the ML API layer for a disaggregated inference system. You will work across inference APIs, model integration, request-routing services, and Cerebras runtimes to deliver...
$176.5k per year
...About The Team And Role Core Infrastructure builds and operates the foundations... ...content that powers our products and our inference‑driven features. The core of the... ...will join an existing team of Senior, Staff, and Principal engineers building the Content Library and...Long term contractFor contractorsLocal areaHome officeFlexible hours- ...customers include some of the world’s largest airlines and ground handling providers. For more information, visit As a Staff Software Engineer, Infrastructure (Cloud), you will own the data infrastructure and cloud systems that power AeroVect's autonomous driving platform....Long term contract
- ...industry-leading training and inference speeds; over 10 times faster... ...monitor, and optimize cloud infrastructure supporting CI workloads,... ...increase developer velocity and engineering productivity. Identify... ...professional experience in software engineering, infrastructure...Long term contract
- ...customers. Cohere is a team of researchers, engineers, designers, and more, who are all... ...building the next generation of agentic AI infrastructure at Cohere. This team sits at the intersection... ...in emerging ML infrastructure, edge inference, or browser‑native modelsOpen‑source...Work at officeLocal areaRemote workHome office
- About the Role Software engineering is undergoing its biggest transformation in decades.AI agents... ...the next.We’re looking for a Senior Staff Engineer to help define what that... ...pipelines to agent sandboxes, testing infrastructure, repository architecture, and AI-native...Long term contractInternshipWork at officeLocal area3 days per week
$180k - $360k per year
...Baseten powers mission-critical inference for the world's most dynamic... ...AI research, flexible infrastructure, and seamless developer tooling... ...and help build the platform engineers turn to ship AI products.About... ...contributions to LLM inference engines and frameworks such as vLLM,...Flexible hours$180k - $360k per year
...Baseten powers mission-critical inference for the world's most dynamic... ...AI research, flexible infrastructure, and seamless developer tooling... ...and help build the platform engineers turn to ship AI products.... ...enhancing the performance of software systems, particularly in the...Flexible hours$165k - $330k per year
...BasetenBaseten powers mission-critical inference for the world's most dynamic AI companies... ...By uniting applied AI research, flexible infrastructure, and seamless developer tooling, we... ...Capital. Join us and help build the platform engineers turn to ship AI products. THE ROLEAs...Flexible hours$181k - $241k per year
Staff Software Engineer, Mobile (App Infrastructure)Remote Canada Affirm is reinventing credit to make it more honest and friendly, giving consumers the flexibility to buy now and pay later without any hidden fees or compounding interest. App Infrastructure is the team...Work at officeRemote workFlexible hours- Shape AI inference software at Cerebras Systems with Python, model serving, and production API... ...accelerator backends.This Senior Software Engineer role focuses on making disaggregated... ...service, framework, runtime, or infrastructure boundariesFamiliarity with REST, gRPC,...
- ..., shaping architecture, and mentoring engineers to do their best work and advance their... ...into how we build, test, and operate infrastructure software. This is not a 'stay in your swim lane... ...GPU infrastructure for training and inference. Here's what your benefits package...Local areaWorldwideFlexible hours
- ...industry-leading training and inference speeds; over 10 times faster... ...of secure, scalable cloud infrastructure and identity platforms in AWS... .... Partner across Security, Engineering, and Infrastructure teams to... ...People who are serious about software make their own hardware. At...
- ...world impact converge at scale. We’re hiring a Staff Backend Engineer to build and operate the infrastructure those models depend on. You’ll design and operate... ...the performance, reliability, and scalability of inference systems. Join us and help influence how billions...Work at officeWorldwide
- Transform the AI experience at Cerebras Systems as a Software Engineer on our Inference Platform team. Shape next-gen architecture while deploying... ...distributed systems • Experience with Kubernetes and cloud infrastructure • Strong knowledge of backend languages like Go or C++ •...
$165k - $330k per year
...BasetenBaseten powers mission-critical inference for the world's most dynamic... ...AI research, flexible infrastructure, and seamless developer... ...and help build the platform engineers turn to ship AI products.... ...THE ROLEAs an Infrastructure Software Engineer at Baseten, you'll...Flexible hours$165k - $330k per year
...Baseten Baseten powers mission-critical inference for the world's most dynamic AI... ...By uniting applied AI research, flexible infrastructure, and seamless developer tooling, we enable... ...Capital. Join us and help build the platform engineers turn to ship AI products. Baseten...Long term contractFlexible hours- As the Senior Staff Software Developer, Developer Infrastructure, you are the technical driver of this entire org — the Uber Tech Lead unifying DevX, TestX,... ...pipelines to agentic coding systems that multiply what each engineer can ship This is a rare opportunity to shape not...Long term contractRemote work
$147k - $196.6k per year
...We are seeking a highly skilled and motivated Staff Software Developer to join our Diagnostics Infrastructure Software team in the Autonomous Vehicle Integrity... .... Your work will be conducted in a collaborative engineering environment, where you will exercise the highest...Long term contractWork at office- Cerebras Systems is hiring a Software Engineer to productionize and optimize our GPU serving stack. You will work across custom inference APIs, vLLM, PyTorch, ROCm, and rack-scale AMD GPU infrastructure to ensure reliability, numerical correctness, observability, and high performance...
$145k - $260k per year
About VGSVGS is building the trust infrastructure for a new era of commerce. As commerce becomes... ...takes exceptional people. Trusted Engineering That Matters Engineering at VGS VGS... ...reliability practices. About the RoleAs a Staff Infrastructure Engineer, you will serve...Work at officeLocal areaImmediate startRemote workFlexible hoursShift work$180k - $210k per year
...applications and next steps. Our partner is looking for a Staff Network Engineer (AI Infrastructure) based in Canada. The Staff Network Engineer will... ...fabrics for large-scale AI workloads. You will develop software-driven solutions that configure network switches, maintain...Local areaRemote workFlexible hours$200k - $250k per year
..., automating everything from engineering workflows to regulatory filings... ...at the heart of how critical infrastructure is planned, monitored, and... ...The RoleWe're looking for a staff engineer to define and drive... ...building and shipping production software, including experience...Work at office$140k - $190k per year
...development of digital investigative software that acquires, analyzes, and shares evidence... ...We’re looking for a seasoned Staff Software Engineer who thrives on technical depth and... ....Nice To Have Experience with Infrastructure as Code (Terraform, Helm, Argo).Experience...Work at officeLocal areaFlexible hours- # Staff Software EngineerJuniper Square190k–235k USDFull-timeremote· leadCanada, USApythonnodejavareactawsdockerkubernetesListed... ....About your roleAs a Staff Software Engineer at Juniper Square, you are a technical... ...equivalent), Docker, Kubernetes, and infrastructure as code-Strong understanding of CI/CD,...Full timeTemporary workWork at officeRemote work
Do you want to receive more vacancies?
Subscribe and receive similar vacancies to Staff Software Engineer, Inference Infrastructure. Be the first to apply!
Related searches
- software trainer Winnipeg, MB
- software qa Winnipeg, MB
- software implementation project manager Winnipeg, MB
- junior software developer .net Winnipeg, MB
- software asset management analyst Winnipeg, MB
- golang software developer Winnipeg, MB
- spécialiste assurance qualité logiciel Winnipeg, MB
- software application developer Winnipeg, MB
- software support Winnipeg, MB
- software intern Winnipeg, MB

