Senior Software Engineer, Sre And Production Engineering - Dgx Cloud

This position is listed on behalf of a partner company, who manages all applications and next steps. Our partner is looking for a Senior Software Engineer, SRE and Production Engineering - DGX Cloud based in Canada.Join a team building and operating large-scale GPU infrastructure that powers demanding artificial intelligence workloads. As a Senior Software Engineer specializing in Site Reliability Engineering (SRE) and Production Engineering, you will develop the automation and operational systems that transform bare-metal hardware into reliable, production-ready cloud services. You will work hands‑on with advanced GPU servers, data processing units, firmware, Linux, and Kubernetes across cloud partner and on‑premises environments. Your work will improve infrastructure provisioning, hardware validation, system recovery, and cluster lifecycle management. You will also help strengthen production reliability by investigating complex failures and turning recurring incidents into lasting automated solutions. This remote opportunity offers the chance to contribute to next‑generation AI infrastructure at significant scale.Accountabilities
Responsibilities
Develop automation for bare‑metal server provisioning, hardware validation, firmware and software upgrades, repairs, and cluster lifecycle management.Build tools that interact with Baseboard Management Controller (BMC) and Redfish interfaces to monitor hardware health, manage server states, and support recovery workflows.Operate and advance NVIDIA NVL72 systems and BlueField‑3 or newer Data Processing Units (DPUs) across cloud partner and on‑premises environments.Diagnose and resolve complex failures involving servers, DPUs, GPU and CPU systems, networking, Linux, and Kubernetes.Convert recurring infrastructure issues into automated detection, troubleshooting, and repair processes.Define validation procedures and production handoff criteria to ensure new infrastructure capacity is introduced safely and consistently.Participate in on‑call rotations, incident response, root‑cause investigations, and follow‑up activities to implement permanent corrective solutions.Collaborate with hardware, networking, platform engineering, data center operations, and partner teams to resolve issues across organizational boundaries.Improve infrastructure reliability, operational consistency, and automation across the production fleet.Requirements
At least 12 years of experience building software for or operating production infrastructure, including substantial hands‑on bare‑metal experience.Strong programming skills in Go or Python, with a proven track record of delivering production automation and services.Direct experience using BMC and Redfish interfaces for server provisioning, hardware health inspection, power management, or fault diagnosis.Practical experience working directly with NVIDIA GPU hardware, particularly NVL72 systems, and BlueField‑3 or newer DPUs.Strong knowledge of Linux, firmware and driver management, network boot processes, and server lifecycle operations from initial provisioning through repair.Experience maintaining production reliability through on‑call responsibilities, incident response, observability, and implementation of durable solutions.Ability to troubleshoot complex issues across hardware, host operating systems, networking, and distributed services.Strong communication skills and demonstrated ownership of technical problems involving multiple teams.Bachelor's or Master's degree in Computer Science, or equivalent practical experience.Additional Experience That Would Be An Advantage
Experience operating BlueField DPUs in DPU mode, including host‑to‑DPU connectivity and lifecycle troubleshooting.Familiarity with NVLink, InfiniBand, Spectrum‑X, or GPU cluster performance validation.Experience designing safe and repeatable workflows for rack‑scale deployment, firmware upgrades, hardware replacement, and customer handoffs.Knowledge of Kubernetes, GitOps, Argo CD, Service Level Objectives (SLOs), and fleet‑wide automation.Benefits
Fully remote work arrangement in Canada.Full‑time position.Competitive compensation package with eligibility for equity.Access to employee benefits and wellness programs.Opportunity to work hands‑on with advanced GPU infrastructure and large‑scale AI workloads.Exposure to complex production engineering challenges across cloud and on‑premises environments.Collaboration with specialized teams working across hardware, networking, platform engineering, and data center operations.Opportunity to contribute to the development and operation of next‑generation AI infrastructure.
#J-18808-Ljbffr