Lead Hpc Systems Engineer

Our client is looking for an engineer with high-performance computing (HPC) and site-reliability experience to keep their growing multi-cloud AI infrastructure reliable and efficient. You will investigate hard operational problems across compute, accelerators, workload scheduling, storage, and networking, and turn what you learn into proactive detection, diagnosis, and safe automated remediation, including AI-powered agents.Success here starts with understanding how HPC infrastructure behaves in production and learning unfamiliar systems fast. Prior experience building or orchestrating AI agents is welcome but not required.Req# 1103823066Responsibilities
Build a working understanding of multi-cloud HPC architecture (clusters, schedulers, accelerators, storage, networking, authentication) and keep it current as the infrastructure changes
Find reliability and performance problems proactively, correlating logs, metrics, and events across cloud providers to build a clear incident picture and spot gaps in telemetry
Diagnose complex issues end to end across Linux hosts, containers, schedulers, distributed workloads, storage, networking, and accelerators, isolating root cause rather than treating symptoms
Turn proven investigations into repeatable tooling and agent-enabled workflows for detection, diagnosis, and bounded remediation, with safety built in: scoped permissions, precondition checks, human approval where needed, rollback, and post-change verification
Strengthen observability and monitoring, measure whether the automation actually works (accuracy, recovery time, recurrence, false positives), and partner with infrastructure, research, security, and cloud-provider teams to turn incident lessons into durable fixes
Requirements
Operations experience with production HPC, GPU/accelerator clusters, or large-scale distributed compute in a reliability-focused engineering role
Strong Linux and networking skills, debugging failures across processes, containers, storage I/O, DNS, connectivity, and authentication
Correlates telemetry across multiple cloud environments using tools such as Prometheus, Grafana, OpenTelemetry, or centralized logging, rather than relying on a single dashboard
Writes and maintains automation in a language such as Python or Go, and has owned complex production investigations from symptom through root cause to a verified, lasting fix
Nice to have
Hands-on with HPC scheduling or cluster orchestration (Slurm, Kubernetes, or similar), and an understanding of how infrastructure health affects workload placement and completion
Supporting AI training or accelerator-intensive workloads: GPU/TPU health, distributed-job failures, interconnect problems, data-loading bottlenecks, or checkpointing
Experience across clouds such as AWS, GCP, CoreWeave, or OCI and their native observability services. You do not need every provider named here
HPC storage and networking (NFS/POSIX, Lustre, object storage, RDMA/InfiniBand or RoCE), infrastructure-as-code and deployment tooling (Terraform, Helm, cluster operators), and failure-injection or replayable incident testing
Building, integrating, evaluating, or orchestrating AI agents for infrastructure operations (tool interfaces, permissions, verifying agent actions)
Welcome, but not required;
strong HPC operations and SRE skills are the foundation of this role
We offer
Extended Healthcare with Prescription Drugs, Dental and Vision, and Healthcare Spending Account (Company Paid)
Life and AD&D Insurance (Company Paid)
Employee Assistance Program (Company Paid)
Telehealth (Company Paid)
Short-term Disability (Company Paid)
Long-Term Disability
Paid Time Off (including vacation and sick days)
Registered Retirement Savings Plan (RRSP) with Company match
Maternity/Parental/Adoption Leave Top-up
Employee Stock Purchase Program
Critical Illness Insurance
Employee Discounts
Unlimited access to LinkedIn learning solutions
EPAM Systems, Inc. is an equal opportunity employer. We recognize the value of diversity and inclusion in creating success for our customers, business partners, shareholders, employees and communities. We are committed to recruiting, hiring, developing and promoting employees without discrimination. As a global employer, this commitment includes complying with all laws in the countries in which we operate. Nevertheless, we believe equal employment practices should not be limited to what the law requires. Equal opportunity and inclusion are essential to motivate, empower and recognize the best in everyone.At EPAM, employment actions are based on individual qualifications, without regard to race, color, religion, creed, gender, pregnancy status, sexual orientation, gender identity, gender expression, marital or familial status, national origin, ancestry, genetics, age, disability status, veteran status, citizenship status when otherwise legally able to work, or any other characteristic protected by law.
#J-18808-Ljbffr