Senior Software Development Engineer (m/f) (H/F)

Responsibilities

Design and ship end-to-end features spanning data models, APIs, background workers, and UI.

Build scalable systems for running, scoring, aggregating, and comparing agent and model evaluation trials.

Improve benchmark task quality using verifiers, rubric-based grading, and LLM-as-a-judge evaluation.

Integrate new models, agents, and benchmarks through generalizable systems rather than one-off implementations.

Maintain performance and reliability through efficient queries, safe migrations, and load-resistant concurrency.

Review pull requests and help maintain consistent engineering standards, tests, and error handling.

Requirements

• 5+ years of experience building and operating production backend systems, including ownership of at least one service from design through on-call.
• Bachelor’s degree or higher in Software Engineering, Computer Science, or a related field.
• Strong Python, Django or a similar framework, PostgreSQL, and task-queue experience such as Celery.
• Hands-on experience with LLM evaluation, including benchmark construction, agent-trial scoring, and result aggregation and comparison.
• Ability to identify ambiguous instructions, weak or gameable verifiers, leaked answers, and invalid checks.
• Comfort working with TypeScript and React when frontend changes are needed.
• Experience with sandboxed or containerized agent environments such as Docker, Kubernetes, Modal, or Daytona is preferred.
• Experience with LLM-as-a-judge or rubric-based grading, coding agents, MCP, distributed scheduling, concurrency limits, database locking, or open-source evaluation and infrastructure projects is preferred.

Benefits

Work at the frontier of AI with leading AI labs and contribute to datasets, reinforcement-learning environments, and benchmarks.

Opportunities to contribute to AI research and showcase work at conferences such as ICLR, ICML, and NeurIPS.

Collaborate with colleagues with experience at Google, Meta, Amazon, and other leading technology companies.

Startup-paced environment with significant ownership and exposure to enterprise AI applications.

The posting states base salary plus equity.