Software Engineer- Inference Platform
$180k - $360k per yearBaseten
About Baseten
Baseten powers mission-critical inference for the world's most dynamic AI companies, like Cursor, Notion, OpenEvidence, Abridge, Clay, Gamma, and Writer. By uniting applied AI research, flexible infrastructure, and seamless developer tooling, we enable companies operating at the frontier of AI to bring cutting-edge models into production. We're growing quickly and recently raised our $1.5B Series F, led by Altimeter Capital, Conviction Partners, and Spark Capital. Join us and help build the platform engineers turn to ship AI products.About Baseten
Baseten powers mission-critical inference for the world's most dynamic AI companies, like Cursor, Notion, OpenEvidence, Abridge, Clay, Gamma, and Writer. By uniting applied AI research, flexible infrastructure, and seamless developer tooling, we enable companies operating at the frontier of AI to bring cutting-edge models into production. We're growing quickly and recently raised our $1.5B Series F, led by Altimeter Capital, Conviction Partners, and Spark Capital. Join us and help build the platform engineers turn to ship AI products.THE ROLE
We're looking for distributed systems engineers and product-minded generalists to build the distributed runtime that powers large-scale LLM inference on Baseten. Our inference platform empowers customers to deploy and operate cutting-edge models with industry-leading performance, scalability, and reliability. It also powers Model APIs, our hosted endpoints for the latest open-source models. You'll work across the stack, from the developer experience customers use to deploy models, through the libraries behind features like tool calling and reasoning, down to the systems that orchestrate deployments on Kubernetes and route traffic efficiently. Your job is to make sure every model on our platform is fast, reliable, and cost-efficient. You'll join a small, high-impact team at the intersection of distributed systems, model performance, infrastructure, and product, helping define how developers use AI models at scale. This role is ideal for engineers who enjoy owning systems in production, solving hard integration problems, and making complex infrastructure simple and reliable for users.EXAMPLE INITIATIVES
The Baseten Inference Stack at NVIDIA Dynamo Day2x faster inference with KV cache-aware routingHow Baseten multi-cloud capacity management (MCM) unifies deploymentsResponsibilities
Build the infrastructure and orchestration systems that deploy and run large-scale distributed LLM inference, including routing, autoscaling, scheduling, and runtime management.Design, build, and operate Model APIs, with a focus on advanced inference capabilities: structured outputs (JSON mode, grammar-constrained generation), tool/function calling, and multimodal serving.Implement platform fundamentals such as API versioning, validation, usage metering, quotas, and authentication.Instrument deep observability (metrics, traces, logs) and build repeatable benchmarks for speed, reliability, and quality. Help set best practices for testing, release automation, and operational excellence.Debug and harden complex production systems spanning Kubernetes, distributed runtimes, networking, and GPU workloads to improve reliability and scalability.Partner with Inference Performance engineers and other teams to make new optimizations broadly available to customers and easy to configure.Own projects end to end, from architecture through deployment, monitoring, and iteration on customer feedback. Along the way, make thoughtful tradeoffs between performance, reliability, operational simplicity, and developer experience.Requirements
Bachelor's, Master's, or Ph.D. in Computer Science, Engineering, or a related field, or equivalent practical experience.3+ years building and operating distributed systems, backend infrastructure, or large-scale APIs where reliability, latency, and scale are first-class concerns.A proven track record of owning low-latency, reliable backend services, including rate limiting, auth, quotas, metering, and migrations.Infrastructure instincts with a feel for performance: profiling, tracing, capacity planning, and SLO management.Comfort debugging performance and reliability issues across multiple layers of the stack, from application behavior down to runtime and infrastructure internals.A strong sense of developer experience. You think about how systems are used, not just how they work.Eagerness to learn new languages, frameworks, and systems, and a real interest in inference engineering. Prior ML or LLM experience isn't required, though experience with model serving or inference systems is a plus.Excellent written communication and collaboration skills, including writing clear design docs and working across functions.NICE TO HAVE
Experience with or contributions to LLM inference engines and frameworks such as vLLM, SGLang, TensorRT-LLM, TGI, or Dynamo.Deep Kubernetes experience, including operators and custom resources, plus familiarity with service meshes or API gateways.Experience with distributed scheduling, autoscaling, or service orchestration.Experience operating GPU workloads in production.A background in developer-facing infrastructure or APIs, or contributions to open-source infrastructure or ML systems.Familiarity with observability tooling, CI/CD systems, or release automation.Benefits
Competitive compensation, including meaningful equity(U.S. only) 100% coverage of medical, dental, and vision insurance for employee and dependentsFlexible PTO policy including company wide Winter Break (our offices are closed from Christmas Eve to New Year's Day!)Paid parental leaveFertility and family-building stipend through Carrot(U.S. only) Company-facilitated 401(k)Exposure to a variety of ML startups, offering unparalleled learning and networking opportunities.At Baseten, we are committed to fostering a diverse and inclusive workplace. We provide equal employment opportunities to all employees and applicants without regard to race, color, religion, gender, sexual orientation, gender identity or expression, national origin, age, genetic information, disability, or veteran status.We are an Equal Opportunity Employer and will consider qualified applicants with criminal histories in a manner consistent with applicable law (by example, the requirements of the San Francisco Fair Chance Ordinance, where applicable).Compensation Range: $180K - $360K
#J-18808-Ljbffr
Baseten powers mission-critical inference for the world's most dynamic AI companies, like Cursor, Notion, OpenEvidence, Abridge, Clay, Gamma, and Writer. By uniting applied AI research, flexible infrastructure, and seamless developer tooling, we enable companies operating at the frontier of AI to bring cutting-edge models into production. We're growing quickly and recently raised our $1.5B Series F, led by Altimeter Capital, Conviction Partners, and Spark Capital. Join us and help build the platform engineers turn to ship AI products.About Baseten
Baseten powers mission-critical inference for the world's most dynamic AI companies, like Cursor, Notion, OpenEvidence, Abridge, Clay, Gamma, and Writer. By uniting applied AI research, flexible infrastructure, and seamless developer tooling, we enable companies operating at the frontier of AI to bring cutting-edge models into production. We're growing quickly and recently raised our $1.5B Series F, led by Altimeter Capital, Conviction Partners, and Spark Capital. Join us and help build the platform engineers turn to ship AI products.THE ROLE
We're looking for distributed systems engineers and product-minded generalists to build the distributed runtime that powers large-scale LLM inference on Baseten. Our inference platform empowers customers to deploy and operate cutting-edge models with industry-leading performance, scalability, and reliability. It also powers Model APIs, our hosted endpoints for the latest open-source models. You'll work across the stack, from the developer experience customers use to deploy models, through the libraries behind features like tool calling and reasoning, down to the systems that orchestrate deployments on Kubernetes and route traffic efficiently. Your job is to make sure every model on our platform is fast, reliable, and cost-efficient. You'll join a small, high-impact team at the intersection of distributed systems, model performance, infrastructure, and product, helping define how developers use AI models at scale. This role is ideal for engineers who enjoy owning systems in production, solving hard integration problems, and making complex infrastructure simple and reliable for users.EXAMPLE INITIATIVES
The Baseten Inference Stack at NVIDIA Dynamo Day2x faster inference with KV cache-aware routingHow Baseten multi-cloud capacity management (MCM) unifies deploymentsResponsibilities
Build the infrastructure and orchestration systems that deploy and run large-scale distributed LLM inference, including routing, autoscaling, scheduling, and runtime management.Design, build, and operate Model APIs, with a focus on advanced inference capabilities: structured outputs (JSON mode, grammar-constrained generation), tool/function calling, and multimodal serving.Implement platform fundamentals such as API versioning, validation, usage metering, quotas, and authentication.Instrument deep observability (metrics, traces, logs) and build repeatable benchmarks for speed, reliability, and quality. Help set best practices for testing, release automation, and operational excellence.Debug and harden complex production systems spanning Kubernetes, distributed runtimes, networking, and GPU workloads to improve reliability and scalability.Partner with Inference Performance engineers and other teams to make new optimizations broadly available to customers and easy to configure.Own projects end to end, from architecture through deployment, monitoring, and iteration on customer feedback. Along the way, make thoughtful tradeoffs between performance, reliability, operational simplicity, and developer experience.Requirements
Bachelor's, Master's, or Ph.D. in Computer Science, Engineering, or a related field, or equivalent practical experience.3+ years building and operating distributed systems, backend infrastructure, or large-scale APIs where reliability, latency, and scale are first-class concerns.A proven track record of owning low-latency, reliable backend services, including rate limiting, auth, quotas, metering, and migrations.Infrastructure instincts with a feel for performance: profiling, tracing, capacity planning, and SLO management.Comfort debugging performance and reliability issues across multiple layers of the stack, from application behavior down to runtime and infrastructure internals.A strong sense of developer experience. You think about how systems are used, not just how they work.Eagerness to learn new languages, frameworks, and systems, and a real interest in inference engineering. Prior ML or LLM experience isn't required, though experience with model serving or inference systems is a plus.Excellent written communication and collaboration skills, including writing clear design docs and working across functions.NICE TO HAVE
Experience with or contributions to LLM inference engines and frameworks such as vLLM, SGLang, TensorRT-LLM, TGI, or Dynamo.Deep Kubernetes experience, including operators and custom resources, plus familiarity with service meshes or API gateways.Experience with distributed scheduling, autoscaling, or service orchestration.Experience operating GPU workloads in production.A background in developer-facing infrastructure or APIs, or contributions to open-source infrastructure or ML systems.Familiarity with observability tooling, CI/CD systems, or release automation.Benefits
Competitive compensation, including meaningful equity(U.S. only) 100% coverage of medical, dental, and vision insurance for employee and dependentsFlexible PTO policy including company wide Winter Break (our offices are closed from Christmas Eve to New Year's Day!)Paid parental leaveFertility and family-building stipend through Carrot(U.S. only) Company-facilitated 401(k)Exposure to a variety of ML startups, offering unparalleled learning and networking opportunities.At Baseten, we are committed to fostering a diverse and inclusive workplace. We provide equal employment opportunities to all employees and applicants without regard to race, color, religion, gender, sexual orientation, gender identity or expression, national origin, age, genetic information, disability, or veteran status.We are an Equal Opportunity Employer and will consider qualified applicants with criminal histories in a manner consistent with applicable law (by example, the requirements of the San Francisco Fair Chance Ordinance, where applicable).Compensation Range: $180K - $360K
#J-18808-Ljbffr
Vacancy posted 2 hours ago
Similar jobs that could be interesting for youBased on the Software Engineer- Inference Platform in Winnipeg, MB vacancy
- ...Cerebras to deliver industry-leading training and inference speeds; over 10 times faster than GPU-based... ...inference. About the RoleWe're hiring a Software Engineer to help contribute to projects on our Inference Platform team. Our team primarily owns the orchestration...Software
- Transform the AI experience at Cerebras Systems as a Software Engineer on our Inference Platform team. Shape next-gen architecture while deploying ultra-fast machine learning services to boost performance and scalability. Cerebras Systems seeks a Software Engineer with 3+...Software
- ...deliver industry-leading training and inference speeds; over 10 times faster than... ...decode on the Cerebras Wafer-Scale Engine. We are hiring a Software Engineer to productionize and... ...systems knowledge across accelerator platforms. Experience modifying or contributing...Software
$165k - $330k per year
About BasetenBaseten powers mission-critical inference for the world's most dynamic AI companies, like Cursor, Notion, OpenEvidence,... ...Conviction Partners, and Spark Capital. Join us and help build the platform engineers turn to ship AI products. THE ROLEAs an Engineer on the...SoftwareFlexible hours$180k - $360k per year
...Baseten Baseten powers mission-critical inference for the world's most dynamic AI... ...Spark Capital. Join us and help build the platform engineers turn to ship AI products.About Baseten... ...Proficiency in enhancing the performance of software systems, particularly in the context of...SoftwareFlexible hours- Staff + Senior Software Engineer, InferenceAbout AnthropicAnthropic’s mission is to create reliable... ...AI systems. About the roleOur Inference team is responsible for building and maintaining... ...AI hardware running in multiple cloud platforms. Inference systems are highly...SoftwareLong term contractWork at officeWorldwideVisa sponsorshipFlexible hours
- ...deliver industry-leading training and inference speeds; over 10 times faster than... ...decode on the Cerebras Wafer-Scale Engine. We are hiring a Software Engineer to build and evolve the ML... ...routing services and Cerebras inference platform to deliver a consistent experience...Software
- Shape AI inference software at Cerebras Systems with Python, model serving, and production API experience across accelerator backends.This Senior Software Engineer role focuses on making disaggregated inference accessible, reliable, and consistent. You will work across model...Software
- This role is for a highly experienced Back-End Software Engineer who wants to work on core systems that shape how the platform (Model Vault) operatesYou will help define the... ...-the-shelf solutionsModel Vault is Cohere’s inference platform for deploying and operating models securely...SoftwareRemote work
$165k - $330k per year
About Baseten Baseten powers mission-critical inference for the world's most dynamic AI companies, like Cursor, Notion, OpenEvidence... ...Partners, and Spark Capital. Join us and help build the platform engineers turn to ship AI products. Baseten powers mission-critical...SoftwareLong term contractFlexible hours- ...industry-leading training and inference speeds; over 10 times... ...are looking for an Inference Platform SDET to join the Inference Service... ...infrastructure, and the software stack that serves the world's... ...an excellent opportunity for engineers who enjoy infrastructure, automation...SoftwareInternship
$165k - $330k per year
...Baseten Baseten powers mission-critical inference for the world's most dynamic AI... ...Spark Capital. Join us and help build the platform engineers turn to ship AI products.About Baseten... ...across the entire fleet.This is a hands-on software engineering role with ownership from...SoftwareInternshipFlexible hours- ...workloads, using Go, controllers, and infrastructure software to operate cloud platforms reliably.As a Software Engineer on Cloud Platform Engineering, you will own... ...upgrades, and GPU capacity. The work supports AI inference infrastructure across providers.Key Responsibilities...SoftwareInternship
$165k - $330k per year
About BasetenBaseten powers mission-critical inference for the world's most dynamic AI companies, like Cursor, Notion, OpenEvidence,... ...Conviction Partners, and Spark Capital. Join us and help build the platform engineers turn to ship AI products. THE ROLEThe AI ecosystem is...SoftwareFlexible hours- ...Cerebras Systems builds the world's largest AI chip and is hiring a Software Engineer to build and evolve the ML API layer for a disaggregated inference system. You will work across inference APIs, model integration, request-routing services, and Cerebras runtimes to deliver...Software
- ...-performing culture. The RoleThe AI Platform Software Engieer will design, build, and operate... ...role sits at the intersection of software engineering, cloud infrastructure, MLOps, and... ...model training, evaluation, deployment, inference, monitoring, and lifecycle management....SoftwareFlexible hours
- # Product Manager, Inference PlatformUnited States2 hours agoID 5413304Price on request## DetailsEmployment... .... Cohere is a team of researchers, engineers, designers, and more, who are all... ...follow from those calls. You own the platform around that endpoint as well, from the console...
- ...We build, deliver and manage a portfolio of data management platforms and mobile offerings in support of our core businesses. We thrive... ...’s next generation Healthcare Informatics platform. As a Software Engineer, you must possess world class technical skills and a strong sense...SoftwareLong term contractLive InRemote workFlexible hours
- ...deliver industry-leading training and inference speeds; over 10 times faster than... ...decode on the Cerebras Wafer-Scale Engine. We are hiring a Senior Software Engineer to build and evolve the ML... ...-specific features into the serving platform. Extend inference services and their...Software
- Drive quality assurance for the Cerebras Inference Platform as a Senior SDET. In this role, you'll focus on infrastructure reliability... ...orchestrate complex systems Requirements: • 3+ years in software engineering or QA/quality engineering • Proficient in Python and/or Go...Software
$111k - $160k per year
...their greatest potential.Title and Summary Senior Software Engineer (Sanctions Service & AI Inference)Who is Mastercard? Mastercard is a global technology... .... This is an opportunity to work on a highly visible platform while helping shape the adoption of AI-enabled...SoftwareInternshipWorldwide- ...deliver industry-leading training and inference speeds; over 10 times faster than... ...centers. As the fleet expands, the software used to monitor and manage it must... ...it. We are seeking a senior software engineer to build the platforms and tools that help our teams understand...Software
- Select how often (in days) to receive an alert:Title: Senior AI Platform Software Engineer Requisition ID: 276518 Join a purpose driven winning... ...workloads, including training, evaluation, deployment, inference, monitoring, and lifecycle management.Own the technical design...SoftwareFlexible hours
$165k - $330k per year
...Baseten Baseten powers mission-critical inference for the world's most dynamic AI... ...Spark Capital. Join us and help build the platform engineers turn to to ship AI products.About Baseten... ...and who enjoy working across product, software development, performance engineering, and...SoftwareInternshipFlexible hours- ...deliver industry-leading training and inference speeds; over 10 times faster... ...Core Infrastructure team builds the software systems that power engineering workflows across Cerebras. Our infrastructure... ...tools, and reusable software platforms that allow engineers to build, test...Software
$200k - $255k per year
...Commercial auto insurers and fleet software companies, including industry... ...Insurance, depend on our platform to access GPS, speeding, and vehicle... ...the roleWe're looking for an engineer who thrives on building... ...Orchestration and workflow engines (Temporal, Step Functions)....SoftwareInternshipWork at officeFlexible hours- ...dataset of empirical information via a revolutionary cloud-based platform to authoritative figures in commercial, environmental, and... ...of hardware design, manufacturing, data processing, and software engineering, our office is a truly inspiring mix of experts from a variety...SoftwareOngoing contractFull timeWork at officeLocal areaRemote workWorldwideHome office3 days per week
- ...are broken. They rely on multiple outdated platforms and error-prone manual work to admit... ...looking for a high-ownership Senior Platform Engineer to build and scale the infrastructure,... ...a related technical field 5+ years of software engineering experience with a strong focus...SoftwareRemote workFlexible hours
$187k - $250k per year
...the industry’s most intriguing cities, united by a drive to push the boundaries of storytelling. Learn more.The Principal Software Engineer (Platform) provides the team with deep technical insight into the details of building modern production pipelines using USD and Houdini...SoftwareLong term contractTemporary workWorldwide- ClearGov is looking for a Software Engineer to join our team. You'll be a key contributor in designing, developing, and scaling the next generation of our cloud-based applications. You’ll work across multiple product lines, integrating acquired technologies, improving performance...SoftwareFull timeWork at officeLocal area
Do you want to receive more vacancies?
Subscribe and receive similar vacancies to Software Engineer- Inference Platform. Be the first to apply!
Related searches
- software engineer part time Winnipeg, MB
- software engineer Winnipeg, MB
- software design engineer Winnipeg, MB
- software engineer - computer Winnipeg, MB
- software developer internship no experience Winnipeg, MB
- ingénieur logiciel Winnipeg, MB
- software engineer co-op Winnipeg, MB
- software development engineer Winnipeg, MB
- software developer Winnipeg, MB
- software engineer - ai machine learning Winnipeg, MB

