Ai Coding Model Evaluation Engineer

Evaluate AI-generated code at Turing using Python, TypeScript, or another production language. Work with coding agents across real-world repositories and assess technical quality.This independent contractor role suits experienced, hands-on software engineers with five or more years of experience. You will review agent behavior, compare solutions, identify failure modes, and create evaluation signals that improve coding models. The work includes collaboration with researchers and engineers.Key Responsibilities:
Evaluate AI-generated code across real-world software repositoriesReview agent behavior, tool usage, and code changesIdentify technical errors and recurring model failure modesCreate rubrics and evaluation criteria for coding tasksProduce preference data and actionable research findingsRequirements:
Five or more years of hands-on software engineering experienceStrong Python, TypeScript, JavaScript, Go, or equivalent proficiencyExperience working in substantial real-world codebasesStrong code-review skills and technical judgmentExperience using modern LLMs or AI coding toolsApply software engineering judgment, code review, and clear communication to improve Turing’s AI coding model evaluations.
#J-18808-Ljbffr