Software Engineer, Infrastructure & Platform

Responsibilities

• Design and build sandboxed environments where AI models can safely execute code, use tools, interact with services, and complete complex tasks.
• Build backend services and infrastructure for large-scale, repeatable AI and agentic evaluations.
• Develop agent scaffolding and evaluation harnesses with tool-use loops, context management, retries, state management, token budgets, and multi-agent workflows.
• Provision and orchestrate isolated environments using Docker, Kubernetes, virtual machines, and cloud infrastructure.
• Design secure networking, permissions, secrets, credentials, and resource isolation for model-driven environments.
• Develop APIs, internal tools, and automation for creating and running evaluations.
• Improve reliability and reproducibility through logging, observability, snapshotting, debugging tools, and automated testing.
• Build systems that run thousands of evaluation tasks reliably and capture relevant artifacts and telemetry.
• Partner with analysts, red teamers, and domain experts to translate evaluation concepts into technical systems.
• Investigate failures across the evaluation stack and distinguish model limitations from infrastructure, harness, or environment failures.

Requirements

• 3–5+ years of professional software engineering experience, particularly in backend, infrastructure, platform, SRE, or distributed systems engineering.
• Strong programming skills in Python and experience building production-quality software.
• Experience designing and operating backend services, APIs, or distributed systems.
• Hands-on experience with Docker, Kubernetes, virtual machines, or other container/orchestration technologies.
• Experience with AWS, GCP, or similar cloud infrastructure.
• Strong understanding of Linux systems, networking, authentication, permissions, and infrastructure security.
• Experience with infrastructure-as-code or automation tools such as Terraform.
• Strong debugging skills across application, infrastructure, and networking layers, especially in agentic loops.
• Ability to build reproducible, observable, scalable, and secure systems.
• Interest in AI systems, agentic workflows, AI security, or model evaluations; prior professional AI experience is helpful but not required.
• Experience with developer platforms, test infrastructure, sandboxes, ephemeral compute environments, agent frameworks, LLM APIs, tool-calling systems, distributed task execution, queues, workflow orchestration, or large-scale automated testing is a plus.
• Familiarity with AI safety, adversarial testing, model evaluations, autonomous-agent systems, Model Context Protocol, agent benchmarks, and AI-agent security risks is a plus.

Benefits

• Professional development support for conferences, continuing education, or leadership training.
• Fully remote work environment for U.S.-based employees.
• Comprehensive health, dental, and vision coverage.
• Generous PTO and paid holiday schedule.
• 401(k) plan.
• Performance-based annual bonus.